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ABSTRACT 

The Words Work! early literacy initiative, in Ramsey Action 
Program (RAP) Head Start centers, was initiated to prepare children to be 
successful third-grade readers and mathematicians. These reports [document 
includes Year Two Analysis and Words Work! Three-Year Evaluation Report, 
September 1999-May 2002] reflect the standardized test results collected in 
years 1 and 2 for the first cohort of Words Work! children from four RAP Head 
Start centers who recently completed first and second grade, respectively, in 
St.' Paul, Minnesota public schools. Three comparison groups were identified 
for the evaluations: (1) non-Words Work! RAP Head Start children in 10 

centers; (2) children who were on the RAP waiting list; and (3) a random 
sample of non-Head Start children. Test scores were adjusted based on the 
average demographic characteristics of children across all study groups. An 
analysis of variance model was used to compare the group means. Based on this 
model. Words Work! children, on average, performed at a higher level than 
those in all three comparison groups in reading, mathematics, and language 
arts. Mathematics appeared to be the strongest subject for Words Work! 
children, followed by reading and language arts. The third-year evaluation 
report notes continued performance of subjects at higher levels than 
comparison groups, and also evaluates specific program characteristics. The 
findings from the study suggest that Head Start programs that utilize Words 
Work! literacy-rich environments can make a difference. The report * s three 
appendices include information on ethnicity and language characteristics of 
each study group, examples of adjusted scores, and confidence intervals. 
(Contains 28 footnotes across the 3 reports.) (Author/KB) 
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ABSTRACT 

The Words Work! early literacy initiative, in Ramsey Action Program (RAP) Head Start centers, 
was initiated three years ago to prepare children to be successful third grade readers and 
mathematicians. This report reflects the standardized test results collected for the first cohort of 
Words Work! children who recently completed first grade in St. Paul Public Schools. Three 
comparison groups were identified for this study: non-Words Work! RAP Head Start children, 
children who were on the RAP waiting list, and a random sample of non-Head Start children. 
Test scores were adjusted based on the average demographic characteristics of children across all 
study groups. An analysis of variance model was used to compare the group means. Based on 
this model. Words Work! children, on average, performed at a higher level than all three- 
comparison groups in Reading, Mathematics, and Language Arts. Mathematics qipeared to be 
the strongest subject for Words Work! children, followed by Reading and Language Arts. The 
results from this study suggest that Head Start programs that utilize Words Work! literacy rich 
environments can make a difference. 



INTRODUCTION 

The long-term outcome of the Words Work! early literacy initiative is to prepare children to be 
successM third grade readers and mathematicians ^ In order to measure this outcome, the St. 
Paul Foundation contracted with the Action Consulting and Evaluation Team (ACET, Inc.) to 
perform longitudinal analysis of children’s test scores over the course of five years. These scores 
will be obtained to address the following research questions: 

• What is the performance level of Words Work! children compared to non-Words Work! 
Ramsey Action Programs (RAP) Head Start children? 

• What is the performance level of Words Work! and non-Words Work! children compared to 
non-Head Start children and children who were on the RAP Head Start waiting list? 

This report reflects test results collected for the first cohort of Words Work! children who have 
recently completed first grade. Test scores will be analyzed for two additional cohorts over the 
next four years. The evaluation team will first describe the research design and sampling 
procedures, next address the two research questions posed above, and then conclude with 
implications of the study, plans for future analyses, and report limitations. 

The Research Design 

The evaluation team was provided a master list of all RAP Head Start children from the RAP 
Head Start office. There are currently 14 RAP Head Start centers, four Words Work! and ten 
non- Words Work! centers^. Most of the RAP Head Start graduates were enrolled in St. Paul 
public schools, while others entered charter schools or neighboring district schools. Only 



^ The four centers were chosen to best represent children enrolled at RAP Head Start. 
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children enrolled in St. Paul Public Schools were tracked because of the uniform administration 
of the Metropolitan Achievement Test, Seventh Edition (MAT7) . The design includes 
following three cohorts of Words Work! children from third through fifth grade, given the cohort 
being followed (see Table 1 below)^. 



Table 1: Year in the Longitudinal Design 



^'cal• 19')‘)-(»0 


2000-01 


2(101-02 


2002-03 


2003-04 


2004-05 


2005-06 


Age 




Grade Level - St Paid Public Schools 




6 Words Work! 


K 


1 


2 


3 


4 


5 


5 Words Work! Words Work! 


K 


1 


2 


3 


4 


4 Words Work! Words Work! Words Work! 


K 


1 


2 


3 



Sampling Procedures 

Three comparison groups were identified as critical references to evaluate the academic 
performance of WordsWork! children^. 

• Group 1: Non- Words Work! RAP Head Start children; 

• Group 2: Children on the RAP Head Start waiting list; and 

• Group 3: A sample of non-Head Start children in St. Paul Public Schools. 

Group 1 children were eligible and received RAP Head Start services. Group 2 children were 
eligible but had never received RAP Head Start services. Both of these groups were simlar to 
WordsWork! children in background characteristics (i.e., primarily students of color with limited 
English from families with low incomes). The last comparison group provided a sample in the 
same age category that had not participated in RAP Head Start to represent typical St. Paul 
Public Schools’ children. 

During the first year of Words Work!, 604 children graduated from RAP Head Start. Of those, 
163 (27%) were from WordsWork! and 441 (73%) from non-WordsWork! centers. A total of 
146 children were on the RAP Head Start waiting list. The evaluation team located 72 percent of 
the Words Work! children enrolled in 42 of the 64 elementary schools and centers with first 



^ Three charter schools in St. Paul used the same standardized test as St. Paul Public Schools. These schools, 
however, did not test children in first grade. The evaluation team received authorization fi-om these schools to pull 
test scores, if available, for subsequent years. 

* This table can be read as follows: For children who participated in Words Work! and are currently six years old, 
most of them recently completed First grade in St. Paul Public Schools (as of June 2002). The evaluation team will 
follow them through fifth grade. Words Woric! children who are currently five years old will enter First grade next 
year (2002-2003). The team will follow them through fourth grade. Finally, Words Woik! children who are 
currently four years old will enter kindergarten in 2002-2003. The team will foUow them until third grade. Length 
of membership with Words Work! (i.e., first year, second year, third ye^) will be coded. 

^ The evaluation team did not use a predecessor group (i.e. cohort of children fi'om the year prior to the st^ of ^ 
Words Work!) in the study siuce such a comparison group would be comprised of a different group of children in a 
different year. Thus, analyzing their performance differences may simply be a product of those two factors. 
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grade programs in the St. Paul Public Schools^. For the comparison groups, 53 percent of non- 
Words Work! children were located in 53 schools and 55 percent of children on the RAP waiting 
list were found in 34 schools. A sample of 300 children that were not associated with RAP Head 
Start was randomly drawn from 50 St. Paul Public Schools’. Table 2 below shows the sample 
size distributions for each study group. 

Table 2. Sample Size Distributions 



1 Studv Group 


Cohort 


Located 


Percent of Cohort 


Number of Schools | 


Words Work! 


163 


118 


?2% 


42 


Non-Words Work! 


441 


234 


53% 


53 


Wait List 


146 


80 


55% 


34 


Non-Head Start 


3144 


300 


10% 


50 



Once children were located, their MAT? scores were extracted from the St. Paul Public Schools’ 
database* *. The MAT? has been reviewed for its adequacy as a measure of academic school 
success and its appropriateness for use in this longitudinal research. The MAT? is primarily a 
norm-referenced test designed to provide normative information on performance of students 
nationwide. Three tests were used for the study: Reading, Mathematics, and Language Arts^. 

• Reading — Measures age-appropriate reading skills including word recogmtion, reading 
vocabulary, and reading comprehension. The reading comprehension test is a group- 
administered silent reading inventory designed to provide an estimate of functional reading 
levels. Each test consists of a series of graded reading passages ranging from well below 
grade-level to well above grade-level, followed by comprehension questions. 

• Mathematics - Assesses basic concepts of number and units, shapes, and money, as well as 
age-appropriate mathematic items that address problem solving and other mathematical 
procedures. 

• Language Arts - Measures age-appropriate language skills in the areas of prewriting, 
composing, and editing. 

Of the 1 18 Words Work! children located, 106 (90%) had test scores available; of 234 Head 
Start children, 21? (93%) had scores; and of the 80 waitlist children, 68 (85%) had scores. 

All 300 children drawn from the St. Paul database had MAT? scores available. 



® Within these distributions, only six schools contained five or more children who participated in Words Work! 

’ The sample size of 300 was selected to create a group approximately the size of the entire Head Start cohort 
(Words Work! plus non-Words Work!) and to provide for the potential mobility of children to retain a large enough 
comparison cohort without over-powering the analyses with a much larger group. This group will be retained over 
the next four years for continuity in the longitudinal analyses. 

* St. Paul Public Schools administers the MAT? in the spring of each year to all students enrolled in grades one 
through ten as part of its accountability system. Exemptions are granted for students with moderate to severe 
disabilities, while other special accommodations, such as limited English proficiency, are provided where 
appropriate. Scores of students who take the MAT? with special accommodations were not released by St. Paul 
Public Schools. 

’ The reading test has the highest reliability index score of 0.95, followed by mathematics at 0.88, and then language 
arts at 0.83. 
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In addition to test scores, the following available demographic characteristics were extracted for 
each study group: 

• Gender • Primary disability 

• Race/ethnicity • Special education instructional setting 

• English Language Learner indicator • Economic indicator 

• Home primary language • Title I student indicator 

These variables were selected by both RAP Head Start staff and the evaluation team to determine 
their relevance in helping explain differences in MAT7 performance*®. Table 3 provides the 
distribution of children in the study groups for each demographic characteristic. As the table 
displays, RAP Head Start children included more children of color, children who participated in 
free or reduced limch, and children with limited English proficiency. 

Table 3: Distribution of Demographic Characteristics across Study Groups 



Demographic 




\\ ord.^NN Orkl 


Non-\N oi*(ls\\ ork! 


\\ aillist 


Non- 11 call Start 




N = 


118 


234 


80 


300 


Gender 


Female 


65% 


53% 


44% 


42% 




Male 


36% 


47% 


56% 


58% 


Children of Color* 


Yes 


94% 


94% 


91% 


63% 




No 


6% 


6% 


9% 


37% 


Limited English* 


Yes 


64% 


75% 


56% 


32% 


No 


36% 


25% 


44% 


68% 


Special Ed 


Yes 


4% 


3% 


9% 


11% 


No 


96% 


97% 


91% 


89% 


Free/Reduced Lunch 


Yes 


97% 


93% 


94% 


64% 




No 


3% 


7% 


6% 


36% 


Title I 


Yes 


86% 


89% 


86% 


76% 




No 


14% 


11% 


14% 


24% 



♦ Appendix A contains the number and percent of each ethnicity and language by study group 



To account for the impact of these variables on test scores, the methodology employed was a 
multivariate analysis of variance to evaluate mean differences based on study group: Words 
Work!, non-WordsWork! Head Start, Head Start waiting list, and non-Head Start. The 
performance of these four groups was compared by computing adjusted scores; these scores were 
based on the average demographic characteristics of children across all study groups**. 



A large body of research has repeatedly demonstrated the importance of variables that embody economic, 
language, and in some cases, gender differences. For a review, see Haladyna, T. M. (2002). Essentials of 
standardized achievement testing: Validity and accountability. Boston, MA: Allyn & Bacon. See particularly 
Chapter 9: Standardized achievement testing of students at risk, pp. 172-190. 

" The differences between each group based on adjusted scores provided more accurate portrayals of differences in 
performance than those found using observed scores. See Appendix B for a complete description of adjusted means. 
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RESULTS 



Research questions: 



• What is the performance level of Words Work! children compared to non-Words Work! 
Ramsey Action Programs (RAP) Head Start children? 

• What is the performance level of Words Work! and non-Words Work! children compared to 
non-Head Start children and children who were on the RAP Head Start waiting list? 

Words Work! children performed statistically higher in Reading and Mathematics than non- 
Words Work! children. Words Work! children scored 13 Normal Cxu^^e Equivalent (NCE) 
points above non-WordsWork! children in Reading, 15 more in Mathematics, and 10 more in 
Language Arts*^. The differences were smaller between WordsWork! and non-Head Start 
children, with Words Work! children scoring seven NCE points higher in Reading, 12 in 
Mathematics, and four in Language Arts. Words Work! children did score statistically higher in 
Mathematics than non-Head Start children. 

The strongest subject for Words Work! children was Mathematics, followed by Reading and 
Language Arts. Mean performance scores can be seen in Table 4, based on adjusted NCE 
scores*^. Statistically significant differences between Words Work! and the other comparison 
groups are noted with an asterisk (Appendix # displays the confidence intervals for each test). 

Table 4. Adjusted (for Demographics) Group Mean Performance for Each Test 



90% Confidence Interval 



Test 


Study Group 


Mean 


Error 


Lower 


Upper 


Reading 


WordsWork! 


543 


3.6 


483 


60.1 


Non- Words Work!* 


41.7 


2.7 


37.3 


46.1 




Waitlist 


42.7 


3.6 


36.8 


48.6 




Non-Head Start 


47.6 


1.7 


44.9 


50.3 


Mathematics 


WordsWork! 


57.7 


4.8 


49.8 


65.6 




Non-Words Work!* 


42.9 


3.6 


36.9 


48.9 




Waitlist 


41.9 


4.9 


33.9 


49.9 




Non-Head Start* 


45.6 


2.2 


41.9 


49.3 


Language Arts 


WordsWork! 


49.2 


3.8 


42.9 


55.5 




Non- Words Work! 


38.9 


2.9 


34.2 


43.6 




Waitlist 


40.4 


3.9 


34.0 


46.8 




Non-Head Start 


44.9 


1.8 


42.0 


47.8 



* Statistically significant at 0.10. 



NCE scores are on interval level measurements whereas percentiles are ordinal and they are the scores typically 
reported by St. Paul Public Schools (as can be seen on their website). NCE scores also have a slight interpretive 
advantage over scaled scores because the national average is 50 - a clear and easily defined reference point for 
comparison purposes. 

These are not actual observed scores, but scores that account for differences due to demographic differences. 
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On average, Works Work! children performed at a higher level than children in the three 
comparison groups. This trend was similar on all three tests, with WordsWork! children 
performing at the highest level, non-Head Start children performing slightly below WordsWork! 
children. Head Start children performing below both of those groups and waitlist children 
performing at the same level as Head Start children (see Figure 1 below). 

Figure 1. Group Performance Based on Adjusted NCE Mean Differences (the ruler 
extends plus or minus 10 NCE points around the average) 



Average 
First Grade 
Readers and 
Mathematicians 
(NCE=50) 





Average Words Work! 
Children (NCE=54-58) 



Average Non-Head Start 




on-Words Work! 



Children (NCE=42-43) 



Individual centers did not contribute significantly to student performance beyond what was 
found based on WordsWork! and non-WordsWork! status. 
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CONCLUSIONS 

Words Work! has improved children’s performance as anticipated in the Reading and 
Mathematic sections of the MAT7. In all three subject areas of the MAT7, Words Work! 
children performed at a higher level than non-Words Work! Head Start children, statistically 
higher for Reading and Mathematics. Mathematics appeared to be the strongest subject for 
Words Work! children, followed by Reading and Language Arts. Within all the study groups. 
Words Work! children performed at the highest level, followed by non-Head Start children, and 
then by non- Words Work! Head Start and Head Start waiting list children who performed at 
similar levels. The results from this study suggest that Head Start programs that receive Words 
Work! literacy rich environments can make a difference. 

Future Analyses 

The analyses completed to date were group comparisons based on the first grade perfomance of 
the first Words Work! cohort. Future analyses vvill include additional cohorts, time points, and 
an evaluation of the impact of Words Work! given the level of implementation and role of the 
family home visitor component. 

Additional Cohorts - The strength of these differences vvill be evaluated over time and in future 
cohorts of Words Work! children. This vvill provide greater evidence of the lasting impact of 
Words Work! participation. As each Words Work! cohort progresses through the St. Paul public 
schools, standardized test data vvill be gathered annually. The second grade students vvill then 
have test scores from first grade and second grade. At this point, the appropriate analytic model 
will be a regression with first-grade scores used as a covariate to adjust for pre-existing 
differences. This will allow the evaluation team to determine the impact of participation in 
Words Work! on growth rates in elementary school performance. Once three years of MAT7 
scores are available, stronger technicjues vvill be employed to estimate growth curves for students 
(e.g.. Hierarchical Linear Modeling) . 

Adoption of Words Work! Practices at Non-Words Work! Centers - Words Work! has 
become a RAP Head Start county vvide initiative. Blending of early literacy programs has 
occurred in 2001-2002 across all Head Start centers. To complete the evaluation of center 
differences in future cohorts, center surveys and site visits have been conducted during the 2001- 
2002 school year to assess the degree of blending and implementation of WordsWork! activities 
(see Spring 2002 Site Visit Report). This blending vvill create more difficulty to assess the 
impact of Words Work!, essentially by eliminating non-Words Work! Head Start centers. 
However, the assessment of the extent of blending vvill provide evidence of the impact of Words 
Work! based on degree of implementation. Through these analyses, the evaluation team will be 
able to assess the relationship between levels of Words Work! adoption and performance. 



'■* Hierarchical linear modeling (HLM) offers several advantages not available in other standard models, including 
the appropriate treatment of missing data (some students may not have MAT? scores at each time point) and 
unbalanced data (there are likely to be more non-Words Woik! children than children in other groups in the 
evaluation follow-along as time progresses). In addition, the statistical techniques available in HLM provide 
superior estimation of effects and relationsHps above current standard regression or analysis of variance techniques. 
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Family Involvement - One of the proposed research questions was about the long-term impact 
of the Words Work! Family Home Visit component. Of those children who received a home 
visit during their participation in Words Work!, is there a relationship between Home 
Involvement scores and MAT7 performance? This question will be evaluated based on an 
analysis of variance model for those children in Words Work! who received home visits. The 
home visitor component, however, was not in effect during the first year of Words Work! 

Family Home Visitor data is available and has been coded for the next cohort of students 
entering first grade in 2002. 

Limitations 

The limitations of this study are similar to others that are not based on experimental studies: 

Participants were not randomly assigned to Words Work! — Nonrandom assignment presents 
challenges in making the argument that the initiative is what made the difference, rather than 
individual characteristics of students. To address this limitation, the evaluation team employed 
five demographic characteristics that have a research base to help account for this difference. 

Impact of individual Schools — The assumption is that some schools may facilitate children s 
academic performance at a higher level than others schools. In order to factor this difference, the 
evaluation team needed an adequate sample of children enrolled at each school. Unfortunately, 
this was not possible given the sample distribution. Thus, school level characteristics could not 
be used in the analyses. 

Small sample sizes in some groups - In an effort to adjust for small sample sizes, the statistical 
tests used in this study may be overly conservative (assuming a small sample fi-om an infinite 
population). There are other statistical adjustments that could be made for finite (small) 
populations that would give us results at a much higher level of significance. However, the 
evaluation team decided to retain the current results to allow for more tentative conservative 
interpretations. 

Missing test scores - There were a small number of Head Start students located in St. Paul 
Public Schools without MAT7 scores or had special testing accommodations. The impact of 
these students is likely to be small. The statistical precision of each group mean presented in this 
report takes into consideration the smaller sample size, recognizing that the students with test 
scores is a sample of the entire cohort that could have taken each test. 

Reliance on MAT7 performance - As reported earlier, the reading test has the highest score 
reliability of 0.95, followed by mathematics at 0.88, and then lan^ge arts at 0.83. These 
reliabilities are fairly high for first grade students, since young children rarely produce reliable 
scores - their performances change dramatically fi"om one testing session to the next. Over time, 
these reliability results improve and scores become more dependable. Although there are strong 
arguments against reliance on standardized testing for assessing performance of young children, 
there are few options in this setting. These results are only one snapshot of the potential impact 
that Words Work! has on participants. 
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Appendix A 

Ethnicity and Language 



Number and Percent of Each Ethnicity by Study Group 





Words Work! 


Non-W orcU W orki 


aillist 


Non-Head Start | 


American Indian 


1 (21%) 


1 (0%) 


3 (4%) 


6 (2%) 


Asian American 


56 (48%) 


151 (65%) 


32 (40%) 


67 (22%) 


Hispanic American 


15 (13%) 


23 (10%) 


10 (13%) 


33(11%) 


African American 


39 (33%) 


45 (19%) 


28 (35%) 


82 (27%) 


Caucasian 


7 (6%) 


14 (6%) 


7 (9%) 


112(37%) 


Total 


118 


234 


80 


300 



Number and Percent of Home Language by Study Group 





Wui cis W ork: 


NOn-\N ords W urk! 


: Waitlist 


Non-Head Start | 


English 


43 (36%) 


57 (24%) 


35 (44%) 


205 (68%) 


Hmong 


53 (45%) 


146 (62%) 


31 (39%) 


52 (17%) 


Other Asian 


3 (3%) 


5 (2%) 


1 (1%) 


11 (4%) 


Spanish 


13 (11%) 


20 (9%) 


9(11%) 


23 (8%) 


Somali 


3 (3%) 


0 (0%) 


0 (0%) 


4 (1%) 


Other 


3 (3%) 


6 (3%) 


3 (4%) 


4 (1%) 


Unknown 


0 (0%) 


0 (0%) 


1 (1%) 


1 (0%) 


Total 


118 


234 


80 


300 
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Appendix B 

Example of Adjusted Scores 

Consider a study of art performance for students who are Left or Right Handed. Consider the 
following table of results reporting mean scores and sample sizes: 



Mean (N) 


Left Handed 


Right Handed 


Girls 


230(10) 


220 (90) 


Boys 


210(40) 


200(10) 



In total, there are 150 students, 50 Left Handed, 100 Right Handed, 100 Girls, 50 Boys. The Left 
Handed Girls performed the best (230); the Ri^t Handed Boys performed the poorest (200). 



The mean score for Left Handed students is 



(230xl0)+(210x40) _^^^ 
10 + 40 



The mean score for Right Handed students is 



(220x90)+(200xl0) _^^g 

90+10 



Notice that Left Handed students scored 4 points lower than Right Handed students, even though 
Left Handed students, whether a Girl or Boy, scored 10 points higher than Right Handed 
students. The discrepancy is because Girls score 20 points higher than Boys, regardless of 
handedness; and Right Handed students are primarily Girls. The benefit of being Left Handed is 
overwhelmed by the deficit of being a Boy, while the deficit of being Right Handed is 
overwhelmed by the benefit of being a Girl. 

There exists a statistical adjustment to correct for such an imbalance, resulting in adjusted means. 
These are basically the same as predicted values from a regression or general linear model. 
Typically, one would enter the mean value of each demographic variable to estimate the mean 
response for all combinations of the variables employed in the model and then take simple means 
over the levels of categorical variable of interest - evaluation group in om case. 

The adjusted means will be a weighted average of the cell means with Girls given the weight of 
100/150 and Boys given the weight of 50/100. 



For Left Handed students, the adjusted mean is 



^230x^^^ 






150. 






210x 






50 

150. 



223.3 . 



For Right Handed students, the adjusted mean is 






210x 






100 

150. 



200 X 



50 

150. 



213.3. 



Notice that the differences in the adjusted means captures the 10 point differences found between 
Left and Right Handed students for each gender, the opposite result of the simple means. 
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Appendix C 
Confidence Intervals 

In cases where the confidence intervals did not overlap, the evaluation team is 90 percent 
confident that the corresponding means are different because of group membership and not 
because of chance sampling^^ Please note that the smaller confidence intervals are a product of 
the larger samples in the non-Head Start and Head Start groups. 

MAT7 - Mathematics 

70 



60 



50 



40 



30 




WordsWork! 



Head Start 



HS Waitlist 



Non Head Start 



The choice in level of confidence is largely arbitrary, but in educational research 90-95% is acceptable 
for evaluating group differences, in an attempt to uncover meaningful differences based on potentially 
useful interventions. By taking a more conservative level of confidence (e.g., 95% or 99%), we run the 
risk of making an error such that a potentially useful intervention is deemed “nonsignificant” - on the 
other hand with a more liberal confidence interval, we run a greater risk of wrongfully arguing an 
intervention is “significanf ’ when the results are actually due to chance sampling, particularly in cases 
where the confidence level is less than 90%. 
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Mat? - Reading 

70 

60' 



50- 
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About ACET 

The Action Consulting and Evaluation Team (ACET, Inc.) is an independent research group 
specializing in the evaluation of PK-16 educational and community-based programs. Located in 
Minneapolis, Minnesota, ACET contracts with a variety of professional consultants to work with 
clients in identifying program strengths and challenges, evaluating goal attainment, and 
providing recommendations for program improvement. Program evaluation activities involve 
both qualitative (e.g. interviews, focus groups, site-visits) and quantitative (e.g. statistical 
analysis, data management, surveys,) analysis. The President of ACET is Stella SiWan Cheung. 

Principal Investigator - Stella SiWan Cheung has consulted with a variety of organizations 
including the Minnesota Department of Children, Families & Learning, the North Central 
Regional Educational Laboratory, and the Minneapolis Foundation. Some of her current 
projects include serving as Principal Investigator for the University of Miimesota Morris New 
Teachers New Technology Program, the Cargill FoundationAJniversity of Minnesota Schools 
First Initiative, and Friendship Community Services. The New York Times and several major 
U.S. newspapers and local radio and television programs have cited her research and 
commentaries. She has presented sessions on evaluation for the U.S. Department of Education, 
the University of Wisconsin at Madison, and the Mmnesota Association of Charter Schools. 

Ms. Cheung was recently announced an Arizona State University College of Liberal Arts and 
Sciences Leader for being a distinguished graduate. She was also named a Woodrow Wilson 
Fellow in 1996. She earned an M.A. in Policy Analysis and Program Evaluation at the Hubert 
H. Humphrey Institute of Public Affairs, University of Minnesota. 

Psychometrician - Michael C. Rodriguez is an Assistant Professor in the Department of 
Educational Psychology at the University of Minnesota. Dr. Rodriguez specializes in 
measurement and assessment, investigating issues related to construct equivalence of test items 
in various formats, classroom assessment practices, and assessment of achievement for students 
with moderate to severe impairments. He has worked extensively 'with schools and school 
districts to develop methods for impro'ving their use of large-scale test information for planning 
and evaluation. He is currently conducting validity studies of several youth development 
program evaluation instruments. He holds a Ph.D. in Educational Measurement and Quantitative 
Methods from Michigan State University. Prior to obtaining his doctorate, he earned an M.A. in 
Public Affairs at the Hubert H. Humphrey Institute of Public Affairs, University of Minnesota. 
Previously, he worked as a Research Associate with the Wilder Foundation Research Center, 
assisting in the development and management of social service and prevention program 
evaluations. 

Site Visit Support - Maureen O’Dougherty specializes in cultural and linguistic diversity. Dr. 

O’ Dougherty conducted a comprehensive evaluation of a federally funded prekindergarten 
program for the New York City Public Schools which had monolingual, bilingual and special 
needs centers. She designed qualitative instruments for classroom observations and staff 
interviews, trained field researchers, analyzed quantitative and qualitative data, and wrote the 
process and final reports. For the American Refugee Center, she designed and conducted an 
assessment of the extent to which higher educational preparation was incorporated in adult 
English Language Learning programs in the Twin Cities. Dr. O’Dougherty earned an M.A. in 
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English as a Second Language from the University of Minnesota and a Ph.D. in cultural 
anthropology from the City University of New York. 

Database Support - Donna Butterbaugh is currently a fifth year Ph.D. student in the 
Psychological Foundations Research Methodology Program at the University of Minnesota. Her 
background includes a B.S. in chemistry, a Masters in teaching chemistry, and seven years of 
teaching high school chemistry and physics. Ms. Butterbaugh has served as a teaching assistant 
for the advanced level courses, as well as both a teaching assistant and instructor for the 
introductory level statistics courses at the University of Minnesota. She also held appointments 
within the Saint Paul Public Schools’ Research, Evaluation, and Assessment Department, the 
Center for Applied Research and Educational Improvement, and the Office of Educational 
Accountability. Her primary research interests include the use of multilevel models for teacher 
and school accountability, and mathematics/science program evaluation. 

Research Support - Jason Butler has consulted with a number of Minnesota charter schools and 
non-profit community organizations as an ACET consultant. Currently, Mr. Butler is working 
on a five-year longitudinal literacy study funded by the Center for the Improvement of Early 
Reading Achievement. His primary interests stem from his experiences as a Pre-school teacher 
in Northern California and include reading comprehension, early literacy development, and 
integration of technology into the classroom. Mr. Butler earned a B.S. in Psychology from North 
Dakota State University and an M.A. in Educational Psychology ^vith a minor in Program 
Evaluation from the University of Minnesota. 
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ABSTRACT 

The Words Work! early literacy initiative, in Ramsey Action Program (RAP) Head Start centers, 
was initiated four years ago to prepare children to be successful third grade readers and 
mathematicians. This report reflects the standardized test results collected for the first cohort of 
Words Work! children who recently completed both first and second grade in St. Paul Public 
Schools. The results summarized in the report represent the second year of a five year study. 
Three comparison groups were identified for this study: non-Words Work! RAP Head Start 
children, cWldien who were on the RAP waiting list, and a random sample of non-Head Start 
children. Test scores were adjusted based on the average demographic characteristics of children 
across all study groups. A m^tivariate analysis of variance model was used to compare second 
grade 2003 group means. Based on this model. Words Work! children, on average, performed at 
a higher level th^ all three-comparison groups in Reading and Mathematics in spring 2003. The 
adjusted mean difference between Words Work! and non-Words Work! children was 
significantly higher, approximately 14 to 17 NCE points. Words Work! children were 
maintaining their lead in 2003. Furthermore, their rate of growth from first to second grade kept 
par with children from the comparison groups. The results fi-om this study reaffirmed first year 
findings and strengthen the suggestion that Head Start programs that utilize Words Work! 
literacy rich environments can make a difference. 

INTRODUCTION 

This report reflects norm-referenced standardized test results collected for the first cohort of 
Words Work! children who have recently completed both first and second grade. The long-term 
outcome of the Words Work! early literacy initiative is to prepare children to be successful third 
grade readers and mathematicians*. In order to measure this outcome, the St. Paul Foimdation 
contracted with the Action Consulting and Evaluation Team (ACET) to perform longitudinal 
analysis of children’s test scores over the course of five years. The results in the report reflect 
the second of a five year study. Test scores will be analyzed for two additional cohorts over the 
next three years. Scores were obtained to address the following three research questions: 

J. What is the performance level of Words Work! children compared to non-Words Work! 
Ramsey Action Programs (RAP) Head Start children? 

2. What is the performance level of Words Work! and non-Words Work! children compared to 
non-Head Start children and children who were on the RAP Head Start waiting list? 

3. What is the performance level of Words Work! and non-Words Work! children over a five 
year period as compared to non-Head Start children and children who were on the RAP 
Head Start waiting list? 

In this report, the evaluation team will describe the research design and sampling procedures, 
address the three research questions posed above, and conclude with implications of the study 
and report limitations. 



1 Words Work! was initiated by the F. R. Bigelow Foundation in 1998. Project funders include the Katherine B. 
Andersen Fund of The Saint Paul Foundation, the F. R. Bigelow Foundation, the Mardag Foundation, The Saint 
Paul Foundation and an anonymous donor. The 
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The Research Design 

The evaluation team was provided a master list of all RAP Head Start children from the RAP 
Head Start office. There are currently 14 RAP Head Start centers, four Words Work! and ten 
non- Words Work! centers^. Most of the RAP Head Start graduates were enrolled in St. Paul 
public schools, while others entered charter schools or neighboring district schools. Only 
children enrolled in St. Paul Public Schools (SPPS) were tracked because of the uniform 
admini stration of a standardized norm-referenced test. The Stanford Achievement Test, Tenth 
Edition (SATIO), was the norm-referenced test administered by SPPS in 2003 to replace the 
Metropolitan Achievement Test, Seventh Edition (MAT?) used previously^. 

The evaluation team follows three cohorts of Words Work! children from third through fifth 
grade (see Table 1 below)'* *. 



Table 1: Year in the Longitudinal Design. 



car 


1999-00 


2000-01 


2mn-02 


2002-03 


2003-04 


2004-05 


2005-06 


Cohort 




Grade Level - St Paul Public Schools 




1 


Words Woik! 


K 


1* 


2* 


3* 


4* 


5* 


2 


Words Work! 


Words Work! 


K 


1 


2* 


3* 


4* 


3 




Words Work! 


Words Work! 


K 


1 


2* 


3* 



♦ Test scores available for analysis 



Prior to 2003, SPPS tested children in grades 1 to 10 in the MAT? reading, mathematics, and 
language arts. First grade MAT? test scores were obtained for the first cohort in spring 2002. 
Currently, only grades 2 to 10 are tested in the SATIO reading and mathematics. Children in 
grades 4, 6, and 8 are also tested in science, social studies, and language. Only second grade 
SATIO test scores were available for the first cohort in spring 2003. In spring 2004, the 
evaluation team will be able to extract third grade scores for cohort 1 and second grade scores 
from cohort 2. 



^ The four centers were chosen to best represent children enrolled at RAP Head Start. 

^ Two charter schools in St. Paul used the same standardized test as St. Paul Public Schools in 2003. These schools, 
however, did not test children in first grade. The evaluation team received authorization from these schools to pull 
test scores, if available and feasible, for subsequent years. 

* This table can be read as follows: For children who participated in Words Work! and are in cohort 1, most of them 
recently completed second grade in SPPS (as of June 2003). The evaluation team will follow them through fifth 
grade. Words Work! children who are in cohort 2 will enter second grade next year (2003-2004). The team will 
follow them through fourth grade. Finally, Words Work! children who are in cohort 3 will enter first grade in 2003- 
2004. The will follow them until third grade. Length of membership with Words Work! (i.e., first year versus 
second year) will be coded. 
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Sampling Procedures 

Three comparison groups were identified as critical references to evaluate the academic 
performance of Words Work! children. 

• Comparison Group 1 : Non-Words Work! RAP Head Start children, 

• Comparison Group 2: Children on the RAP Head Start waiting list; and 

• Comparison Group 3: A sample of non-Head Start children in St. Paul Public Schools. 

Group 1 children were eligible and received RAP Head Start services. Group 2 children were 
eligible but had never received RAP Head Start services. Both of these groups were siimlar to 
Words Work! children in background characteristics (i.e., primarily students of color with 
limited English fi-om families with low incomes). The last comparison group provided a sample 
in the same age category that had not participated in RAP Head Start to represent typical SPPS 

children. 

A majority of children tested in 2002 were located with test scores in 2003 (see Table 2 below). 



Table 2: Sample Size Distributions. 



Stiiilv (iroiip 


cohort 


Locatctl 

2002 


1 C'StC'tl 
2002 


Tested 

2003 


Of those tested in 2002, 
tested in 2003 


Words Work! 


163 


118 


106 


99 


93% 


Non- Words Woik! 


441 


234 


217 


192 


88% 


Wait List 


146 


80 


68 


58 


85% 


Non-Head Start 


3144 


300 


300 


250 


83% 


Total 






691 


599 


87% 



In previous years, SPPS employed the MAT7 for their district assessment program. District 
personnel reviewed the content specifications of several standardized tests and chose to switch to 
the SATIO because of the close alignment with district curriculum in reading and mathematics. 
The SATl 0 is very similar to the MAT7 as it is primarily a norm-referenced test designed to 
provide normative information on performance of students nationwide. 

• SATIO Reading - Measures age-appropriate reading skills including word study skills in 
grades two through four and grade seven, and reading vocabulary and reading 
comprehension across all grades. The reading comprehension test is a group-administered 
silent reading inventory designed to provide an estimate of fimctional reading levels. Each 
test consists of a series of graded reading passages ranging fi*om well below grade-level to 
well above grade-level, followed by comprehension questions. 

• SATIO Mathematics - Includes age-appropriate maAematic items that address problem 
solving and mathematical procedures. Students in grades four through eight are allowed to 
use calculators for the mathematics problem solving portion of the test. 
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SAT 10 scores were extracted from SPPS records, including demographic information to allow 
for changes in ELL, Title 1, and free-reduced lunch status. Table 3 below outlines the 
distribution of demographic characteristics across the four study groups. 

Table 3: Distribution of Demographic Characteristics across Study Groups. 



Demouriipliic 




Words Work: 


Non-W ords 
Work: 


\N aitlist 


Noii-Hc;ul 

Stan 


Total 


Gender 


Female 


65.7% 


57.3% 


46.6% 


43.2% 


51.8% 




Male 


34.3% 


42.7% 


53.4% 


56.8% 


48.2% 


Children of Color 


Yes 


92.9% 


93.8% 


89.7% 


61.6% 


79.8% 




No 


7.1% 


6.3% 


10.3% 


38.4% 


20.2% 


English Language 


Yes 


65.7% 


74.0% 


55.2% 


33.6% 


53.9% 


Learner (ELL) 


No 


34.3% 


26.0% 


44.8% 


66.4% 


46.1% 


Free/Reduced Lunch 


Yes 


96.0% 


94.3% 


93.1% 


61.6% 


80.8% 




No 


4.0% 


5.7% 


6.9% 


38.4% 


19.2% 


Title 1 


Yes 


89.9% 


91.1% 


87.9% 


75.6% 


84.1% 




No 


10.1% 


8.9% 


12.1% 


24.4% 


15.9% 




n = 


99 


192 58 




250 599 





To account for the impact of these variables on test scores, the methodology employed was a 
multivariate analysis of variance to evaluate mean differences based on study group membership. 
Words Work!, non-Words Work! Head Start, RAP Head Start Waiting List, and non-Head Start. 
The performance of these four groups was compared by computing adjusted scores; these scores 
were based on making the demographic characteristics of children across all study groups 
equivalent^. This analytic model was the same as the model employed in 2002 . To calculate 
performance over time, the evaluation team used the MAT7 first grade scores as a covariate (i.e., 
kept MAT7 constant) to calculate SATIO growth from first to second grade. 



^ A large body of research has repeatedly demonstrated the importance of variables that embody economic, 
language, and in some cases, gender differences. For a review, see Haladyna, T. M. (2002). Essentials of 
standardized achievement testing: Validity and accountability. Boston, MA: Allyn & Bacon. See particularly 
Chapter 9: Standardized achievement testing of students at risk, pp. 1 72-190. 

® The differences between each group based on adjusted scores provided more accurate portrayals of differences m 
performance than those found using observed scores. 
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RESULTS 

Please note that direct comparison between 2002 and 2003 NCE points should not be made since 
the two are based on different tests, SAT 10 and MAT7, with different scales. Both tests have 
slightly different definitions of reading and mathematics and employ different samples of 
children to compute national norm-referenced scores (see Lmutations section on Page 1 2). 

The results shown on pages 6 through 8 should be interpreted independently from the results in 
the previous year. To calculate growth from the previous year, the evaluation team used MAT7 
performance as a covariate to predict 2003 SAT 10 scores. Thus, the results from pages 9 and 10 
should be interpreted in a relative manner, with non- Words Work! children as the reference point 
for comparing growth. 

Research questions (2003 Performance): 

1. What is the performance level of Words Work! children compared to non-Words Work! 
Ramsey Action Programs (RAP) Head Start children? 

2. What is the performance level of Words Work! and non-Words Work! children compared to 
non-Head Start children and children who were on the RAP Head Start waiting list? 

Reading 

Words Work! children performed statistically higher in reading than all other groups. 
Specifically, Words Work! children scored 

• 1 6.6 Normal Curve Equivalent (NCE) points above non-Words Work! children, 

• 1 0.9 NCE points above Waitlist children, and 

• 8.5 NCE points above non-Head Start children. ^ 

Mean performance scores are displayed m Table 4, based on adjusted NCE scores . Statistically 
significant differences between Words Work! and other comparison groups are noted with an 
asterisk (Appendix A contains the confidence intervals). 

Table 4: Adjusted (for Demographics) Group Mean Performance for SATIO Reading. 



95% Confidence Interval 



Test 


Study Group 


Mean 


Standard 

Error 


Lower 

Bound 


Upper 

Bound 


Reading 


Words Work! 


53.5 


3.7 


46.3 


60.7 




Non-Words Work! * 


37.0 


2.8 


31.6 


42.4 




Waitlist* 


42.7 


3.8 


35.3 


50.1 




Non-Head Start* 


45.0 


1.8 


41.5 


48.6 



Note. * indicates statistically significant differences from Words Work! using multivariate Least 
Significant Difference (LSD) comparisons at/K0.05. 



’ These are not actual observed scores, but scores that account for differences due to demographic differences. 
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Mathematics 



Words Work! children performed statistically higher in mathematics than non-Words Work! 
children. The adjusted mean differences were smaller (non-sigmficant) between other groups. 
Specifically, Words Work! children scored ^ 

• 1 3.6 NCE points higher (statistically significant) than non-Words Work! children , 

• 7.8 NCE points higher than (non-significant) Waitlist children, and 

• 5.7 NCE points higher (non-significant) than non-Head Start children. ^ 

Mean performance scores are displayed in Table 5, based on adjusted NCE scores . Statistically 
significant differences between Words Work! and other comparison groups are noted with an 
asterisk (Appendix A contains the confidence intervals). 

Table 5: Adjusted (for Demographics) Group Mean Performance for SATIO Mathematics. 



95% Confidence Interval 



Test 


Study Group 


Mean 


Standard 

Error 


Lower 

Bound 


Upper 

Bound 


Mathematics 


Words Work! 


52.9 


4.2 


44.5 


61.2 




Non-Words Work!* 


39.3 


3.2 


33.0 


45.5 




Waitlist 


45.1 


4.3 


36.6 


53.6 




Non-Head Start 


47.1 


2.1 


43.0 


51.3 



Note. * indicates statistically significant differences from Words Work! using multivariate Least 
Significant Difference (LSD) comparisons at/K0.05. 

As can be seen in the standard errors and confidence intervals (Table 5 and Appendix A), 
students in each group varied widely in their performance in mathematics, more so than in 
reading (thus the larger standard errors and wider confidence intervals)*®. Resulting mean 
differences are less likely to be statistically significant, since each mean is not a complete 
representation of the group (because the students within each group vary in their performance in 
a larger range, we are less certain about the mean being a fair representation of the group as a 
whole). 



* NCE scores are on interval level measurements whereas percentiles are ordinal (a simple rank ordering of 
children); NCE points are the scores typically reported by St. Paul Public Schools (as cm be seen on Aeir website). 
NCE scores also have a slight interpretive advantage over scaled scores because the national average is 50 - a clear 
and easily defined reference point for comparison purposes. 

’ These are not actual observed scores, but scores Aat account for differences due to demographic differences. 

A confidence interval is the range of plausible values calculated vwthin some level of error. The standard error of 
a variable is a measure of how far it is likely to be fi-om the expected value (score). The stMdard error for Words 
Work! children in reading was 3.7 and 4.2 in math; the error for non-Words Work! in reading was 2.8 and 3.2 in 
math; error for waitlist children was 3.8 in reading and 4.3 in math; and the error for non-Head Start was 1.8 in 
reading and 2.1 in math. 
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In summary, Words Work! children maintained their lead in 2003. On average, Words Work! 
children performed at a higher level than children in the three comparison groups. Work! 
children scored similarly in both reading and mathematics, with these children performing at the 
highest level, followed by non-Head Start and wait list children, and Head Start children 
perfonhing below all the comparison groups (see Figure 1 below). 



Figure 1: Group Performance Based on Adjusted NCE Mean (the ruler extends plus or 
minus 12 NCE points around the average NCE of 50). 



Average 
Second Grade 
Readers and 
Mathematicians 

(NCE=50) 








Average Words Work! 
Children (NCE=53-54) 




Average Non-Head Start 
_ Children (NCE=45-47) 

Average Wait List 
Children (NCE=43-45) 

Average Non-Words Work! 
Children (NCE=37-40) 



Note: Individual centers did not contribute significantly to student performance beyond what was 
found based on WordsWork! and non-WordsWork! status. 
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Research Question (Growth over Time): 

3. What is the performance level of Words Work! and non-Words Work! children over a five 
year period as compared to non-Head Start children and children who were on the RAP 
Head Start waiting list? 

To address this research question, the evaluation team used MAT7 first grade test scores to 
calculate second grade growth for all groups tested. MAT7 first grade performance was the 
co variate added to the existing multivariate analysis of variance model. This method helps to 
determine how Words Work! children perform in second grade compared to the other groups if 
all children performed at the same level in first grade. 

Reading 

Words Work! children grew at a slightly higher rate in reading compared to all other groups fi-om 
2002 to 2003. In particular. Words Work! children grew 

• 5.1 NCE points above non-Words Work! children (i.e.. Words Work! children increased their 
first grade lead above non- Words Work! children by 5.1 NCE points), 

• 3.3 NCE points above non-Head Start children, and 

• 2.7 NCE points above waitlist children. 

These increases, however, were small and not statistically significant. This suggests that Words 
Work! children are progressing at similar rates with children fi'om the comparison groups. 

Figure 2 below shows the adjusted mean difference for each group once demographic and first 
grade performance are included (see Appendix B for the confidence intervals). 

Figure 2: Adjusted (for Demographics and First Grade Performance) Group Mean 
Difference for Reading relative to non-Words Work! children. 




Non- Words Work! growth was set at zero for comparison purposes 
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Mathematics 

Words Work! children grew at a slightly higher rate in mathematics th^ non- Words Work! 
children from 2002 to 2003. However, Words Work! children grew slightly less than non-Head 
Start add waitlist children. In particular, Words Work! children grew 

• 3.4 NCE points above non-Words Work! children (i.e., Words Work! children increased then- 
first grade lead above non-Words Work! children by 3.4 NCE points), 

• 2 NCE points below non-Head Start children, and 

• 1.6 NCE points below waitlist children. 

These differences again^were small and not statistically significant. This suggests that Words 
Work! children are progressing at similar rates to others from the comparison groups. Figure 3 
below shows the adjusted mean difference for each group once demographic and first grade 
performance are included (see Appendix B for the Confidence Intervals). 

Figure 3: Adjusted (for Demographics and First Grade Performance) Group Mean 
Difference for Mathematics relative to non-Words Work! children. 



O 

U 

O 

Pi 

U 



5 

4 - 
3 - 
2 
1 
0 



Words Work! Non-WW! 



Waitlist Non-Head Start 



Non-Words Work! growth was set at zero for comparison purposes 
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CONCLUSIONS 



The Words Work! Initiative continues to improve children’s performance in Reading and 
Mathematics. For both years, Words Work! children scored significantly higher than non- Words 
Work’ children. The adjusted mean difference between Words Work! and non-Words Work! 
children was large, approximately 14-17 NCE points. Words Work! children performed at the 
highest level, maintaining their lead in 2003, followed by non-Head Start and wait list children, 
and Head Start children below all other comparison groups. The rate of growth for Words 
Work! children is also promising. Words Work! children are able to continue their performance 
at similar rates to children from the comparison groups. This upward trend suggests that the 
Words Work! Initiative continues to impact student performance over a two year period. 



Limitations 

The limitations of this study are similar to others that are not based on experimental studies. 

Change of measurement in.struments from MAT7 to SATIO - The MAT7 and SATIO normative 
scores were not equated with each other. A direct comparison from year to year cannot 1^ made 
between the two tests. The MAT7 and SATIO, however, are both based on common national 
goals and objectives in reading and mathematics. The SATIO was selected by SPPS to replace 
the MAT7 used previously. The district decided that the SATIO is more closely aligned with 
district curriculum. In the long run, this may mean that the SATIO will be more sensitive to 
instruction and more closely measure growth and development in reading and mathematics for 
children in SPPS. 

Participants were not randomly assigned to Words Work! - Nonrandom assignment presents 
challenges in making the argument that the initiative is what made the difference, rather than 
individual characteristics of children. To address this limitation, the evaluation team employed 
five demographic characteristics that have a research base to help account for this difference. 

Impact of individual Schools - The assumption is that some schools may facilitate children’s 
academic performance at a higher level than others schools. This impact may become ^eater as 
children spend more years in specific schools - schools will have a longer period in which to 
make unique impacts upon children in their care. In order to factor this difference, the evaluation 
team needed an adequate sample of children enrolled at each school. Unfortunately, this was not 
possible given the sample distribution. Thus, school level characteristics could not be used in the 

analyses. 

Comparison Group Experiences - Two of the comparison groups constitute unknown 
characteristics, including non-Head Start and waitlist children. The non-Head Start sample was 
selected randomly from SPPS. Children from the waiting list, however, were not selected at 
random. Further, both groups may have participated in another pre-school program or childcare 
center prior to beginning kindergarten. As such, we cannot be sure that both groups did not 
receive some other preschool intervention prior to enrollment. 
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Small sample sizes in some groups — In an effort to adjust for small sample sizes, the statistical 
tests used in this study may be overly conservative (assuming a small sample from an infimte 
population). There are other statistical adjustments that could be made for finite (small) 
populations that would give us results at a much higher level of significance. However, the 
evaluation team decided to retain the current results to allow for more tentative conservative 
interpretations. 

Missing test scores - As the longitudinal analyses progress over time, there will be larger 
numbers of children that cannot be located in St. Paul Public Schools. The impact of these 
children is likely to be small to start with, but will grow over time. The statistical precision of 
each group mean presented in this report takes into consideration the smaller sample size, 
recognizing that children with test scores represents a sample of the entire cohort that could have 
taken each test. Since we retain the same sample of children in the non-Head Start poup, all 
groups have similar probabilities of losing children due to missing test days or moving away 
from the district. 
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Appendix A: 

SATIO 2003 Confidence Intervals 

In cases where the confidence intervals did not overlap substantially, the evaluation team is 95 percent 
confident that the corresponding means are different because of group membership and not because of 
chance sampling. The choice in level of confidence is largely arbitrary, but in educational research 90- 
95% is acceptable for evaluating group differences, in an attempt to uncover meaningful differences 
based on potential useful interventions. Please note that the smaller confidence intervals are a product of 
the larger samples in the non-Head Start and Head Start groups. 






29 



13 



August 26, 2003 



Appendix B: 

SATIO Growth Model Confidence Intervals 

In cases where the confidence intervals did not overlap substantially, the evaluation team is 95 percent 
confident that the corresponding points are different because of group membership and not because of 
chance sampling. The choice in level of confidence is largely arbitrary, but in educational research 90- 
95% is acceptable for evaluating group differences, in an attempt to uncover meaningful differences 
based on potential useful interventions. Please note that the smaller confidence intervals are a product of 
the larger samples in the non-Head Start and Head Start groups. 
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About ACET 

The Action Consulting and Evaluation Team (ACET, Inc.) is an independent research group 
specializing in the evaluation of PK-16 educational and community-based programs. Located in 
the Twin Cities, ACET contracts with a variety of professional consultants to work with clients 
in identifying program strengths and challenges, evaluating goal attainment, and providing 
recommendations for program improvement. The President of ACET is Stella SiWan Cheung. 

Principal Investigator - Stella SiWan Cheung has consulted with a variety of organizations 
including the Minnesota Department of Children, Families & Learmng, the North Central 
Regional Educational Laboratory, and the Minneapolis Foundation. Some of her current 
projects include serving as Principal Investigator for the University of Minnesota Morris New 
Teachers New Technology Program and the Cargill Foundation/University of Minnesota Schools 
First Initiative. The New York Times and several major U.S. newspapers and local radio and 
television programs have cited her research and commentanes. She has presented sessions on 
evaluation for the U.S. Department of Education, the University of Wisconsin at Madison, ^d 
the Minnesota Association of Charter Schools. Ms. Cheung was recently announced an Arizona 
State University College of Liberal Arts and Sciences Leader for being a distinguished graduate. 
She was also named a Princeton University Woodrow Wilson Fellow in 1996. She earned an 
M.A. in Policy Analysis and Program Evaluation at the Hubert H. Humphrey Institute of Public 
Affairs, University of Miimesota. 

Psychometrician - Michael C. Rodriguez is an Assistant Professor in the Department of 
Educational Psychology at the University of Minnesota. Dr. Rodriguez specializes in 
measurement and assessment, investigating issues related to construct equivalence of test items 
in various formats, classroom assessment practices, and assessment of achievement for students 
with moderate to severe impairments. He has worked extensively with schools and school 
districts to develop methods for improving their use of large-scale test information for planning 
and evaluation. He is currently conducting validity studies of several youth development 
program evaluation instruments. He holds a Ph.D. in Educational Measurement and Quantitative 
Methods from Michigan State University. Prior to obtaining his doctorate, he earned an M.A. m 
Public Affairs at the Hubert H. Humphrey Institute of Public Affairs, Umversity of Minnesota. 
Previously, he worked as a Research Associate with the Wilder Foundation Research Center, 
assisting in the development/management of social service and prevention program evaluations. 

Database Support - Steve Culpepper is a graduate student in the Department of Educational 
Psychology at the University of Minnesota vrith an emphasis in research methodology. Steve 
currently assists with the statistical analysis of Words Work!, a longitudinal study examining the 
impact of early literacy programs for children enrolled in Headstart. Steve manages the 
longitudinal database and assists Professor Rodriguez with statistical analysis. 
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Introduction 

This report evaluates the Words Work! program, a $1 .9 million partnership initiated by the F.R. 
Bigelow Foundation between the Saint Paul Foundation, Ramsey Action Program (RAP) Head 
Start and the Miimesota Literacy Council (MLC). The 2001-2002 school year was the third year 
of this multi-year initiative. Approximately 300 children were enrolled from fall to spring of 
each year in four Head Start sites. 

This evaluation report has seven parts: the introduction, executive summary, description of 
participating children, project impact, critical factors, recommendations and conclusion. Since 
this is the third year of evaluation, the report format is slightly different than in previous two 
year-end reports. This year the report includes three-year analyses as well as findings specific to 
the third year of the program. The project impact section of the report is broken down in the 
following way: For each of four categories (Student Skills, Classroom / Instructional 
Techniques, Family Involvement, and Sustainability), an overall three-year summary is 
presented, followed by bulleted major findings for each of the program’s three years, beginning 
with the first year. Case studies are used as sidebars to illustrate the evidence presented. 
Supporting tables and data summaries are included in the appendix. 

Words Work! Goals and Strategies 

Words Work! has two goals: 

• Increase early literacy skills of children to prepare them to be successful readers and learners 
when they are in school; and 

• Increase the family role in the development and delivery of high quality early literacy 
experiences for their children. 

The specific goals for children include: 

• Increase speaking and listening vocabulary; 

• Increase phonological awareness; 

• Develop print awareness; 

• Attempt to print; 

• Learn the letters of the alphabet; and 

• Learn to count from one to 10 and imderstand the meaning of numerals. 

The specific goals for families include: 

• Increase literacy activities in family life; 

• Help family members reinforce classroom goals through home visits; 

• Achieve home environments conducive to the child’s literacy development; and 

• Increase quality parent-child interaction. 
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The strategies Words Work! uses to achieve these goals include: 

• Develop curriculum support services and materials; 

• Train classroom staff members; 

• Provide equipment and materials to support an enhanced literacy environment; and 

• Provide home literacy visitors to: 

• Identify specific parent needs; 

• Develop information, materials, services and other supports that parents need to provide 
enhanced early literacy experiences for their children; 

• Identify ways to share information about early literacy with parents; and 

• Develop information and materials to share with parents. 

Methodology 

This report is based on several sources of information collected from August 1999 to May 2002: 

• Fall and spring student assessment data from three years; 

• Case studies of eight children over three years; 

• Three classroom observations from each of three years from each of the four Words Work! 
sites; 

• Two staff focus group observations from each of the three years; 

• Parent attendance reports and parent meeting notes from 1 999-2000 and 2000-01 ; 

• Fall and spring parent focus group findings from three years for all four sites; 

• Pre- and post-program surveys of participants in the home visitor program from 2001-02; and 

• Fall and spring home visitor focus groups from 2001 -02. 

The data were collected and analyzed by a multi-cultural and -lingual team of six evaluation 
consultants. The multi-cultural and -lingual team assured that parents, children and staff 
members could speak the language of their choice. This has meant that at least three languages 
were used for every parent focus group, fostering diverse participation. Trust more easily grew 
with direct communication, a critical condition for reliable and valid data collection. The team’s 
cultural imderstanding also provided sensitivity and accuracy in analyzing and interpreting the 
data. 

Children were assessed using the SPARK Skills Checklist (a tool produced by the authors of the 
classroom curriculum), a staff-developed writing assessment, a phonological assessment, an 
ABC assessment and the Boehm Test of Basic Concepts, a vocabulary test with national norms. 
The phonological assessment used was modified over the course of the program’s three years to 
tailor it to the developmental needs of the children. 

Many children relocated of left the program throughout the three years of the Words Work! 
program. Only those children who stayed in the program for a complete school year were 
included in that year’s analysis. Student progress was measured using a conditional proportion 
formula to calculate the proportion of children at each center that had room to grow and did grow 
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from the fall of one year to the spring of that academic year (i.e. the percent of children eligible 
to grow in the fall minus the percent of children eligible to grow in the spring divided by fall 
eligibility). 

A detailed summary of the data sources can be foimd in the appendix. 
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EXECUTIVE SUMMARY 



WORDS WORK! 

THREE-YEAR EVALUATION REPORT 
September 1999 - May 2002 

Prepared by Mary Ellen Murphy 
with the assistance of 

Sandy Donovan, Stella SiWan Cheung, Vanessa McKendall, Chia Youyee Vang and 

Aloida Zaragoza 
June 19, 2002 

Introduction 

This report evaluates the Words Work! program, a $1.9 million partnership initiated by the F.R. 
Bigelow Foimdation between the Saint Paul Foundation, Ramsey Action Program (RAP) Head 
Start and the Minnesota Literacy Council (MLC). The 2001-2002 school year was the third year 
of this multi-year initiative. Approximately 300 children were enrolled eaeh year in four Head 
Start sites. 

Words Work! has two goals: 

• Increase early literacy skills of children to prepare them to be successful readers and learners 
when they are in school; and 

• Increase the family role in the development and delivery of high quality early literacy 
experiences for their children. 

This evaluation report provides presents third-year findings as well as three-year analysis of the 
program. It is based on a comparison of student assessment data, classroom observations, parent 
and staff focus groups, case studies, an analysis of parent meetings and feedback on the literacy 
home visitors program. 



Project Impact 



Student Skills 

Words Work! successfully developed many of the desired literacy skills during each school year 
of the program’s three years. At the end of each year, the overwhelming majority of students 
performed well on the SPARK assessments, knew how a book operated, meaningfully counted 
from one to 10 and attempted to write their name. In the third year, approximately 80-90 percent 
of the children mastered these literacy skills by the end of their school year as compared to 70-80 
percent in the first year. The mastery of these skills indicates that each year the program was 
more effective in developing the literacy skills. 

The Words Work! program was less successful in building children’s phonological skills and 
conceptual vocabulary and in helping them to identify the ABCs than it was in improving their 
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SPARK, book operation, counting and name-writing skills. Children’s phonological, conceptual 
vocabulary and ABC skills grew each year but the gains were minimal until the third year. In 
the Spring 2002, most children demonstrated statistically significantly improvements in their 
phonological skills and vocabulary, with two sites making tremendous strides. 

In the second and third years of the program. Words Work! children exceeded the national Head 
Start goal of knowing more than 10 ABCs. The average number of ABCs that Words Work! 
children knew in those years was 13 and 14 respectively. Although these averages exceed the 
national goal, it still fell short of the program goal of 50 percent of the children going to 
kindergarten in the fall knowing all 26 ABCs. In the program’s respective years, only 9 percent, 
13 percent and 25 percent of the prospective kindergarten children knew all 26 ABCs. 

The rate of literacy growth expanded as the program developed, particularly for children of 
color, with accelerated growth in the third year. In the first year, growth was center- and 
classroom-based. Growth in the second year was higher than the first year but remained 
sporadic, still differing by classroom and center. In the third year, growth was high across 
centers except for conceptual vocabulary and rhyming. As noted above, one center and one 
classroom in particular effectively taught their children the vocabulary as measured by the 
Boehm and rhyming. 

Classroom / Instructional Techniques 

The improvement of children’s literacy skills is a reflection of the changing instructional 
techniques of teachers as they continue to increase their understanding of the Spark curriculum 
and their ability to modify it to meet student needs. The program has made tremendous strides 
in developing teachers’ instructional capacities. Over the three years, teachers mastered 
techniques that effectively teach children literacy skills. In addition, as teachers became 
proficient in their techniques, they also became more effective in embedding them in the 
classroom’s natural learning flow. Teachers more often enthusiastically seize teachable 
moments and praise children’s progress. Most have developed strong leadership skills among 
their peers and continue to improve their ability to use evaluation data to modify their 
instruction. They are beginning to share effective teaching strategies across sites and among 
classrooms as they develop a professional learning community. 

The most dramatic change was in the teachers’ attitudes about their role. Originally many saw 
their job as plagued with difficulties brought by the children, but over the next three years 
teachers’ attitudes shifted. They began to see themselves as having the ability to influence 
lea rnin g and the program gave them additional tools to use. Their “tool box” of instructional 
practices and techniques expanded over time, increasing the teacher’s comfort with their role. 
They moved fi"om blaming children and families to demonstrating leadership by looking for 
more and new ways to help children learn. Teachers became intentional in their practices. 
Classroom routines were established that reinforced literacy development. Teachers quickly 
realized that the routines reinforced literacy skills in children. The link between classroom 
practices and instructional techniques to children learning was formalized. Teachers challenged 
themselves and their mentors with finding additional ways to help children learn. Transitions to 
various classroom activities became opportunities to reinforce learning. 
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Family Involvemeiit 



The program successfully increased parents’ involvement in their children’s education and 
family support for literacy development. Family involvement has increased, particularly in the 
area of at-home involvement. Parents reported that they have been using the teachers’ and home 
visitors’ suggestions to support their children’s literacy activities. For three years the Book-in-a- 
Bag program continued to be parents’ most used and favorite school project. For many families 
it provided the only children’s book in their home. A regular place for children’s schoolwork 
has been establish^ in most homes. Parents regularly asked their children about schoolwork 
and reading with the child has increased considerably. While the home environment still lacks 
man y of the rcsouTces (such as children’s books, games, blocks, and writing materials) parents 
began using home materials they do have for teaching their children. Parents attending the focus 
groups continued to request more activities they can do with their children to support their 
learning. 

As children came home proudly demonstrating what they learned and communication from 
teachers increased, parents became increasingly aware of their child’s progress. The information 
was well received by the parents. The more parents knew the more they wanted to know. In the 
third year, parents in some sites asked for written comments from teachers once a week. The 
newsletters were effective ways the school communicated about classroom activities, themes and 
activities families could complete to support their child’s classroom work. Telephone calls, 
verbal reminders when parents picked up their children and bus driver comments were the most 
effective ways the program used to let parents know about upcoming events requiring their 
attendance or participation. Monthly parent meetings changed from the first and second year 
when committee reports were tediously translated into two or three languages, to the third year 
when meetings consisted of demonstrations and parent activities mimicking children’s activities. 
This change dso helped to keep parents well informed. Parents also appreciated increased 
contact with their child’s teacher at the monthly parent meetings. 

Parents with a few exceptions were overwhelmingly pleased with dieir child’s Head Start 
experience and development of literacy skills. For parents with limited English abilities, 
bilingual staff members helped them feel welcome and informed. However, family involvement 
is the least developed part of the program and remains the greatest challenge. Family 
involvement, like effective teaching practices, varied across classrooms. Some teachers are 
adept at communicating with parents and involving families in creative ways. In those 
classrooms, fami ly pictures grace the walls. Family cultural symbols are used as learning 
opportunities. Parents were warmly welcomed and meaningful activities were found for them to 
do in the classroom. Other classrooms minimally involved parents. 

Sustainability 

One of the Words Work! goals was that the program would be sustained by the collaborating 
agencies after the foundation funding ended. Words Work! has made sizable gains in ensuring 
sustainability but sustainability is still questionable. Several factors contribute to the program’s 
continuance at Head Start, including the program’s staff members’ strong commitment to 
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developing children’s literacy and their knowledge of effective instructional techniques. Many 
of the program’s teaching strategies and classroom materials have become part of the non-Words 
Work! classrooms. Recent interviews suggested that competition between program teachers and 
non- Words Work staff members has been reduced and non-Words Work staff members seem 
anxious to learn about the program and replicate the effective teaching strategies. MLC staff 
members have warmly welcomed the program’s home visitors and continued to link the program 
staff with adult literacy resources. 

In spite of the program’s grounding in staff members’ skills, enthusiasm and commitment, the 
challenges are formidable. Head Start’s structure, personnel policies, leadership and 
communication continue to interfere with successful program outcomes and institutionalizing 
Words Work!. The lack of an effective system of accountability is foremost. The inequity 
among classrooms and centers lessened with training and strong support but were not eliminated. 
A low pay scale and a lack of time for teachers to reflect, plan and modify instruction also 
remain barriers. A lack of communication among centers, agency divisions and management 
prevented all staff members from understanding the program, its impact and relationship to their 
job. A lack of leadership and charges of institutional racism have created a strained atmosphere 
of tension and imcertainty. 

The integration of the program into the MLC structure is also challenging. The home visitor 
program is a new venture for MLC. Pre-school children and their families are new target groups. 
MLC continues to question the fit of the Words Work! program with the agency’s focus on adult 
and school-age children’s literacy. 



Critical Factors 

Several key factors affected the program’s ability to achieve its desired goals, including: 

• Instructional practices; 

• Staff development and support; and 

• Institutional practices. 



Recommendations 



The Words Work! program has been particularly adept at addressing recommendations included 
in each year’s evaluation report. Program leadership and center staff members have been quick 
to solve problems and find creative solutions. Unfortunately some challenges still remain. The 
recommendations below are offered to improve children’s literacy development, instructional 
practice, famil y involvement and replicate and sustain the program beyond foundation funding. 
Specific suggestions include: 

• Continue and enhance the same pattern of staff development that includes training, direct 
feedback, mentoring, videotaping, reflection and sharing of best practices; 
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• Clarify teacher expectations of performance and develop a system of accountability to ensure 
quality performance of all teachers, in particular, target areas for improvement with follow- 
up; 

• Increase teacher unders tanding of phonics and its relationship to literacy and reading and 
continue to develop and share effective teaching strategies for phonological skills, 
particularly rhyming, vocabulary and ABCs; 

• Set program standards that outline pre-school literacy skills for children’s success in 
kindergarten; 

• Improve the discipline policies and system, using classroom teacher ideas, to reduce the 
classroom disruption due to children with severe behavior problems; 

• Increase coordination among the home visitor, classroom teachers and family advocates; 

• Continue to develop unique strategies to involve families of varying cultures, including 
increased co mmuni cation in the home language and culturally-appropriate materials; 

• Ensure bilingual teachers with the same languages as the children; 

• Investigate the relationship between family literacy practices and children’s literacy 
development; 

• Replicate best practices and continue to train staff members in non-Words Work! sites; 

• Involve teachers in enrollment decisions; and 

• Develop a strategy to inform and involve new Head Start leaders in developing policies and 
procedures that support the program’s sustainability and children’s literacy development. 



Conclusion 

The Words Work! program has progressed toward achieving its outcomes over the past three 
years. In particular, the program has successfully helped pre-school children master many 
literacy skills, including knowledge of a book’s operation, meaningful coimting, ability to wnte 
their name and vocabulary including colors, opposites and many other language skills. While 
growth in phonological skills, conceptual vocabulary and ABCs was not as great as other literacy 
skills, the growth was sizable and statistically significant. In addition, the growth for children of 
color accelerated and reversed the two-year trend of parallel progress. The long-standing 
achievement gap between children of color and white children began to close this year. 

The accelerated growth and mastery is due to the effective instructional practices of program 
teachers and a literacy-dominated curriculum. Teachers continue to develop new and creative 
ways to teacher children literacy skills. Unfortunately, teaching abilities vary Ifrom classroom to 
classroom and are reflected in differences in children’s mastery of literacy skills. 

F amil y involvement also grew over the three years. The first year of the home-visiting 
component of the program completed this year, resulting in increased family involvement and 
support of their cWld’s literacy development. Home reading increased, the home learning 
environment of participating families became more conducive to literacy development and 
parents’ knowledge of classroom activities increased. Family enthusiasm for the program 
continued to grow, with families often in focus groups offering few suggestions. 
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Challenges remain. Significant institutional barriers continue to interfere with children’s literacy 
development, family involvement and the sustainability of the program. These barriers must be 
addressed. The program must continue to be strengthened in the existing centers, particularly in 
areas of phonological skill, conceptual vocabulary, ABCs and family involvement. At the same, 
incremental replication of Words Work! into non-program sites should continue. 
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DESCRIPTION OF PARTICIPATING CHILDREN 



The children described in this report are those who were continuously enrolled from fall to 
spring. Their numbers ranged from a high of 35 1 students the second year to a low of 281 
students the third year. The following student demographics show how diverse the population is 
by many different criteria. Children differ in ways such as their race and ethnicity, age, English- 
speaking ability, and whether or not they have special needs. In addition, these demographics 
also shifted during each year of the program, reflecting the changing student body at the four 
sites of Baker, Bigelow, McDonough and St. Clair. 

Student testing mobility: Student testing mobility is measured by how many of the children took 
the same skills test in the fall and spring of the academic year. Overall, there was a high degree 
of stability at all four sites during all three years. In 2001-02, St. Clair had the highest level of 
stability with 94 percent of children taking the same test both times in the year. Baker had 81 
percent of its children taking the same tests, and both Bigelow and McDonough had stability 
levels of 79 percent. 

Race /Ethnicity: The following chart shows the race and ethnicity of the Words Work! children 
over the three years of the program. Each year a majority of the children was children of color 
(92 percent the first year, 88 percent the second year and 92 percent the third year). Overall, the 
number of Hispanic children nearly doubled during the program’s three years. The number of 
Hmong or Asian children declined from 41 percent of all children the first year to 31 percent the 
third year. The number and percent of Afiican children (mostly Somali) also increased over the 
three years, although due to record keeping methods, the data does not fully reflect this (during 
the third year. Head Start began counting African children as Afiican- American, thus inflating 
the number of Afiican- American children and deflating the number of Afiican children). From 
their own observations and other evidence, evaluators are confident that in fact the number of 
Afiican children increased over the three years while the number of Afiican-American children 
decreased. 



Race / Ethnicity 


1999-2000 


2000-01 


2001-02 


Total 


Afiican 


27 (8%) 


37 (10%) 


2 (1%)* 


66 (7%) 


Afiican-American 


98 (30%) 


102(29%) 


92 ( 33%)* 


292 (30%) 


American Indian 


2(1%) 


2 (1%) 


4 (1%) 


8 (1%) 


Caucasian 


26 (8%) 


43 (12%) 


21 (8%) 


90 (9%) 


Hispanic 


37(11%) 


59 (17%) 


68 (24%) 


164 (17%) 


Hmong / Asian 


133 (41%) 


108 (31%) 


87 (31%) 


328 (35%) 


Other / Missing 


4(1%) 


0 


7 (2%) 


11 (1%) 


Total 


327 


351 


281 


959 


* During the program’s third year, F 


ead Start began counting Afiican children as 



Afiican-American. 

The largest racial/ethnic group at each site for the school year 2001-02 is listed below: 

• Baker: Hispanic (63 percent), after an increase each of the previous two years; 

• Bigelow: Afiican and Afiican American (56 percent); 
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• McDonough: Asian and Hmong (63 percent); and 

• St. Clair: Hispanic (40 percent), after a slight increase each of the two 
previous years. 

Age: The age breakdown of Words Work! children remained relatively stable over the program’s 
three years. Each year, fewer than 10 percent of the children were 3-year-olds. The remaining 
children were relatively evenly split between 4-year-olds and 5-year-olds all three years. The 
first year, the largest group (50 percent) was 4-year-olds, the second year the largest group (49 
percent) was 5 -year-olds and the third there were about the same number of 4-year-olds and 5- 
year-olds. 



Age 


1999-2000 


2000-01 


2001-02 


Total 


3-year-olds 


22 (7%) 


33 (9%) 


25 (9%) 


80 (8%) 


4-year-olds 


163 (50%) 


148 (42%) 


129 (46%) 


440 (46%) 


5 -year-olds 


142 (43%) 


170(49%) 


127 (45%) 


439 (46%) 


Total 


327 


351 


281 


959 



Gender: Males and females were closely balanced over the program’s three years. The biggest 
contrast between genders was in the third year, when about 53 percent of children were female 
and 46 percent were male. 



Gender 


1999-2000 


2000-01 


2001-02 


Total 


Female 


165 (50.5%) 


171 (48.7%) 


149 (53%) 


485 (51%) 


Male 


162 (49.5%) 


180(51.3%) 


128 (45.6%) 


470 (49%) 


Missing 


0 


0 


4(1.4%) 


4 


Total 


327 


351 


281 


959 



English-speaking Ability: About half of the children were considered fluent English-speakers, 
but the levels of fluency varied somewhat during the program’s three years. The percent of 
fluent children dropped from 5 1 to 46 percent while the percent of children who were “somewhat 
fluent” increased from 9 to 17 percent. The percent of children who had no English also dropped 
over the three years, from 20 to 14 percent. 



English Skills 


1999-2000 


2000-01 


2001-02 


Total 


No English 


64 (20%) 


59(17%) 


39 (14%) 


162(17%) 


Little English 


67 (20%) 


82 (23%) 


59(21%) 


208 (22%) 


Somewhat Fluent 


29 (9%) 


27 (8%) 


47(17%) 


103 (11%) 


Fluent in English 


167 (51%) 


183 (52%) 


129 (46%) 


479 (50%) 


Missing 


0 


0 


7 (2%) 


7 (1%) 


Total 


327 


351 


281 


959 



Enrollment Status: About three-fourths (74 percent) of all Words Work! children were new to 
the program in 2001-02. At St. Clair, only 60 percent of the students were new, at McDonough 
67 percent were new, at Baker 76 percent were new, and Bigelow had a new-student rate of 83 
percent. 
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Disability status: About 16 percent of all Words Work! children had special needs in 2001-02. 
This was a decrease from the previous year, when more than 20 percent had special needs. 
Bigelow had the highest percent of special needs children (20 percent, down from 27 percent the 
previous year). At St. Clair, 19 percent of children were special needs, 13 percent at 
McDonough and 9 percent at Baker. 
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PROJECT IMPACT 



Student Skills 

Words Work! successfully developed many of the desired literacy skills during each school year 
of the program’s three years. At the end of each year, the overwhelming majority of students 
performed well on the SPARK assessments, knew how a book operated, meaningfully counted 
from one to 10 and attempted to write their name. In the third year, approximately 80-90 percent 
of the children mastered these literacy skills by the end of their school year as compared to 70-80 
percent in the first year. The mastery of these skills indicates that each year the program was 
more effective in developing the literacy skills. Interestingly, children’s literacy skills in the 
second and third years of the program were greater when they entered Words Work! than those 
of the first-year children. Improvements in and standardization of the assessment process are the 
likely causes of the difference. Case studies illustrate this mastery of skills: 

Youa is the fourth of six children. The mother does not work outside of the home. 

The father works at a company where he is an assembler. They have been in the 
U.S. for 12 years. Youa ’s mother said that enrolling her daughter at Head Start 
was important in preparing her for kindergarten. She has seen significant 
changes in her child. She stated that her child knew very little English before 
going to Head Start. Toward the middle to the end of the school year, the child 
could speak English with her older siblings. The mother credited her child’s 
ability to count and print her name to Head Start. She said, “Her dad and I 
didn ’t teach her any of this. She knew how to count in Hmong from 1-10, but she 
did not know how to say it in English. It seems that she would learn certain 
things in school, and then older siblings would help her at home. Sometimes the 
kids would tell each other the stories that they were learning at school. [Youa] 
tells her older siblings her stories from Head Start. ” 

The Words Work! program was less successful in building children’s phonological skills and 
conceptual vocabulary and in helping them to identify the ABCs than it was in improving their 
SPARK, book operation, coimting and name-writing skills. Children’s phonological, conceptual 
vocabulary and ABC skills grew each year but the gains were minimal imtil the third year. In 
the Spring 2002, most children demonstrated statistically significantly improvements in their 
phonological skills and vocabulary, with two sites making tremendous strides. In one particular 
classroom of three- four- and five-year olds, the mean score on the rhyming assessment was four 
times higher than the mean in the site’s other three classrooms. One site also taught children 
conceptual vocabulary words, such as imder, above, and in front of so that their scores on the 
nationally-normed test began to compare favorably with their national preschool peers. 

In the second and t hi rd years of the program. Words Work! children exceeded the national Head 
Start goal of knowing more than 10 ABCs. The average number of ABCs that Words Work! 
children knew in those years was 13 and 14 respectively. Although these averages exceed the 
national goal, it still fell short of the program goal of 50 percent of the children going to 
kindergarten in the fall knowing all 26 ABCs. In the program’s respective years, only 9 percent, 
13 percent and 25 percent of the prospective kindergarten children knew all 26 ABCs. 
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The rate of literacy growth expanded as the program developed, particularly for children of 
color, with accelerated growth in the third year. In the first year, growth was center- and 
classroom-based. Growth in the second year was higher than the first year but remained 
sporadic, still differing by classroom and center. In the third year, growth was high across 
centers except for conceptual vocabulary and rhyming. As noted above, one center and one 
classroom in particular effectively taught their children the vocabulary as measured by the 
Boehm and rhyming. 

The Words Work! pogrom also made significant progress toward the goal of preparing students 
for kindergarten. Case studies indicate that many children who had been in the Words Work! 
program were prepared for the demands of first grade: 

Neng is attending HOPE Academy charter school for the 2001-02 academic year. 

She is in first grade. As the youngest child, she receives help from her older 
siblings. Her father works first shift while her mother works the second shift. 

Neng ’s mother said that she had seen great improvement in her daughter ’s ability 
to speak English. She gives her teacher a lot of credit and expresses concerns for 
her own lack of time to be with her children. She stated, “We barely have time to 
eat together once or twice during the entire week There is no time in our life for 
activities, especially during the cold season. ” When asked to recite the letters of 
the alphabet, the child could do so. The evaluator wrote the letters of the 
alphabet and asked the child to read the letters out of order. The child was able 
to recognize all letters of the alphabet. The mother credited her husband in 
addition to the older siblings and her teacher for this progress. When asked what 
she liked most at school, she said that she liked singing songs and coloring. In 
the spring 2002 interview, the child could sound out several words, such as 
“she, ” “he, ” and “go. ” 

Jenny ’s family has lived in St. Paul for eight years. There are three children in 
the family and she is the youngest. Her mother ’s educational background is one 
year of college. Her occupation is housewife/child care. The family has moved 
away before the end of the school year, thus the child will no longer be 
participating in the project. Jenny ’s mother indicated that she enrolled her 
daughter in Head Start to help prepare her for kindergarten. She stated, “So she 
can be around other children. Be in a more learning environment and be ready 
for kindergarten. ” The mother knew her child’s Head Start teacher by name and 
had met with her. Throughout the school year, she used materials provided by 
Head Start to help her child learn, such as SPARK stories, newsletters, and 
sheets. When she had questions, she would contact the Head Start center 
manager. She did, however, express some concern about the center manager 
needing to be more available. What the mother valued most about her child being 
at Head Start is “that she loves being there. She ’s sad when not there. ” 

Additionally, Head Start is “preparing her for the next grade, interacting with 
others... ’’ 
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This was Rosita ’s first year in kindergarten. This year she continues to be 
sociable, inquisitive and very much a communicator. She is more able to express 
her feelings and is also more patient and focused in her attention. Her listening 
and reading skills have also developed more. Her mother reports that she has 
performed very well in school and has benefited from being in a bilingual 
classroom. Her Spanish is clearly stronger than before although she had 
previously been exceptionally bilingual. She has learned more about her two 
cultures and is a proud Latina. Her teacher has helped her to learn to be calmer 
and to improve her behavior. The child is also very proud of her progress and 
excitedly shared the many things she does well at school: reading, drawing and 
coloring inside the lines, “cutting very well”/ “cotando miry bien. ” Without 
prompting, she offered to read to the evaluator at the last interview. She read 
from own book and sounded out letters when she needed to in Spanish. In 
summary, it appears that this child has been prepared well for learning through 
her Head Start experience. It nurtured her love for books and learning and also 
provided the focus she needed, giving her a solid foundation for future learning. 
The fact that she is in a bilingual kindergarten classroom is an added value to 
maximize her learning in two languages. In her own words, Rosita excitedly told 
the evaluator, “The class can be interesting and I can do things very well. ” 

Debbie is one of five children and lives in a two-parent household, where she is 
the youngest. Her mother is a housewife and does eldercare on a part-time basis. 
The mother is a 31 year-old white woman who has lived in St. Paul her entire life. 
Her educational background is completion of 1 1‘^ grade. Debbie finished first 
grade in spring 2001. Her mother reports that she is doing very well. She 
remains sociable and has many friends. She is physically large for her age, which 
her mother feels brings its own challenges. She likes to be heard arui express her 
opinions. She can now count past 100 and likes numbers. She enjoys reading 
and reads everyday. She reads labels, the cartoons in the paper, large print 
books and books from school and the library. In the car, she writes letters and 
now enjoys using her computer skills. But, some reading is difficult for her and 
her mother feels she should be reading at a higher level. 



Annual Findings 
1999-2000 (First Year) 



Strengths: 

• SPARK assessments: At least 75 percent of Words Work! children mastered 23 of the 37 
SPARK literacy skill areas the first year. Growth in these skills was greatest in recognizing 
names in print, coimting, and singing. 

• Book awareness: An overwhelming majority of Words Work! students knew how a book 
operated at the end of this year. The average year-end score for understanding a book’s 
operation was 6.2, out of possible scores ranging fixsm zero to seven. 
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• Attempts to print: The program demonstrated great success getting children to print. All but 
one Words Work! child attempted to print this year. 

Challenges: 

• Phonological awareness: Although children made significant gains in four areas of the 
phonological assessments, assessment results were generally low in this area. Their greatest 
success was in identifying single syllable words and hearing soimds, while rhyming was their 
greatest challenge. 

• Vocabulary: Words Work! children struggled to perform as well on vocabulary assessments 
as their peers around the coimtry. In the program’s first year, 62 percent of Words Work! 
children scored in the bottom quartile on the Boehm assessment. Nine percent scored in the 
top quartile. 

• ABCs: The Words Work program fell short of its goal of 50 percent of children entering 
kindergarten knowing all 26 letters of the alphabet. In the program’s first year, 9 percent of 
students going to kindergarten knew all 26 letters. 



2000-01 (Second Year) 



Strengths: 

• SPARK assessments: At least 75 percent of Words Work! children mastered 30 of the 37 
SPARK literacy skill areas this year. Growth in SPARK assessments was greatest in certain 
activities, including recognizing names in print and pointing to certain colors. 

• Book awareness: The average year-end score for imderstanding a book’s operation was 6.1, 
out of possible scores from zero to seven. 

• Attempt to print: Words Work! made great progress in helping children print. Children 
scored an average of 4.75 on a one- to six-point scale. 

Challenges: 

• Phonological awareness: While Words Work! children on average made statistically 
significant progress from fall 2000 to spring 2001 in at least four of the six areas of the 
Phonological Assessment, they continued to struggle with these skills. Children had their 
greatest success in identifying individual words and hearing soimds; they had the least 
success with rhyming. 

• Vocabulary: While Words Work! children had more success on vocabulary assessment tests 
this year than the previous year, they still struggled. Thirty-three percent of Words Work! 
children scored in the bottom quartile on the Boehm assessment test, while 16 percent scored 
in the top quartile. 

• ABCs: For the second year the project fell short of its goal: In spring 2001, 13 percent of 
children going to kindergarten knew all 26. 
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2001-02 (Third Year) 



Strengths: 

• SPARK assessments: At least 75 percent of Words Work! children mastered 34 of the 41 
SPARK literacy skill areas this year. Growth was greatest in the areas of recognizing names 
in print, naming more than four body parts, and naming a circle. 

• Book awareness: Words Work! children continued to display proficiency in book operation. 
The average year-end score for imderstanding a book’s operation was 6.1, out of possible 
scores from zero to seven. 

• Attempt to print: Most Words Work! children had attempted to print by the end of this year. 
Average scores at three centers ranged from 5.6 to 4.6 on a one- to six-point scale. [One 
center began using a different scale for this assessment and thus its scores could not be 
included in a composite.] 

• Phonological: Overall, Words Work! children made statistically significant progress from 
Fall 2001 to Spring 2002 in all three areas of the phonological assessment. Although children 
did make significant gains in this area, overall success was still relatively low. Children had 
their greatest success in identifying syllables, and they continued to have the least success in 
rhyming. 

Challenges: 

• Vocabulary: Children this year struggled more than children in the program’s second year on 
the vocabulary assessment. Forty-two percent of Words Work! children scored in the bottom 
quartile on the Boehm test, while 15 percent scored in the top quartile. 

• ABCs: For the third straight year, the program fell short of its goal of 50 percent of children 
entering kindergarten knowing all 36 letters. Nevertheless, the percent of children who did 
know all 26 letters jumped to 25 percent in Spring 2002. 
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Classroom / Instructional Techniques 



The improvement of children’s literacy skills is a reflection of the changing instructional 
techniques of teaehers as they continue to increase their imderstanding of the Spark curriculum 
and their ability to modify it to meet student needs. The program has made tremendous strides 
in developing teachers’ instructional capacities. Over the three years, teachers mastered 
techniques that effectively teach children literacy skills. In addition, as teachers became 
proficient in their techniques, they also became more effective in embedding them in the 
classroom’s natural learning flow. Teachers more often enthusiastically seize teachable 
moments and praise children’s progress. Most have developed strong leadership skills among 
their peers and continue to improve their ability to use evaluation data to modify their 
instruction. They are beginning to share effective teaching strategies across sites and among 
classrooms as they develop a professional learning community. 

The process of learning new teaching techniques, using them to teach literacy skills and then 
modifying them based on student progress has become a regular process for most teachers. For 
example, all centers and several individual teachers set literacy goals at the beginning of the third 
year Aat reflected their unique challenges. When goals were not met, they quickly strategized 
and experimented with new techniques. Brainstorming and problem-solving were common 
among many staff members and teachers sought additional support and resources fi'om Head 
Start to improve their outcomes. 

Teachers experienced professional development as a part of the Word Work training and follow- 
along support, and many teaehers also continued their professional growth outside of the centers. 
Several teachers completed their degree programs. In one center, three of the teachers received 
associate degrees and all plan to pursue bachelor degrees. One teacher graduating with a 
master’s degree in education used Words Work! as her dissertation topic. Teachers are 
enhancing their learning both in and out of the classroom. 

This profile of teachers as professional learners contrasts sharply with teacher attitudes during 
the program’s first year. For example, at the opening training sessions, teachers described 
evaluation as being like “going to the dentist” and “burdensome.” Most teaehers feared the 
evaluator’s classroom observations. The feedback sessions with staff where evaluators shared 
the results of observations, focus groups and assessments were primarily filled with complaints 
about what did not work in the program. Most felt overwhelmed, and questioned and doubted 
the program expectations for children, suggesting that they too high. Many teachers raised the 
same questions about the appropriateness of teaching literacy to preschool children that are often 
reflected in the pre-school literature. Teachers feared that program goals would not be achieved 
and many staff members were generally defensive. While some teachers supported the 
program’s goals early on, most viewed the program as something being “done to them.” 
Teaehers noted that most had not been part of the planning and that the assessments were time 
consuming, constituting a seemingly overwhelming burden when teachers were already pressed 
for time. 
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The center managers also became more professional. In the first year, the center manger’s 
primary role was administrative, monitoring compliance with federal regulations and completing 
paperwork with little time for educational leadership. In two centers the center manger also 
taught, making it extremely difficult for parents to contact the center or teachers and leaving 
minimal time for educational coaching. In the third year, two center mangers became mentor 
teachers, freeing them of their classroom duties to concentrate on training and supporting the 
classroom teachers. They suggested ideas to teachers as they observed their classrooms, 
problem-solved when goals fell short, identified areas for additional work and advocated for their 
staff members to the program director and at managerial meetings. Teachers’ attitudes toward 
the mentors switched from seeing them as an extra person to help with children to a resource for 
improving their teaching, a source of fresh ideas, a source of encouragement and classroom 
materials. The center manger and mentor teacher became additional resources in the learning 
community, focused on the needs of the children and program goals. 

For many teachers, evaluators were seen as peers and a resource for reflection and learning. 
Teachers went fi'om complaining about not imderstanding the children’s assessment data reports 
and requesting only bar charts, to in several cases asking for detailed classroom data. In addition 
to improving assessment and reporting techniques, evaluators worked as partners with teachers 
to increase their ability to use data. This year, one center asked to put their children’s data into 
the computers themselves, simplifying record keeping but also allowing them to more quickly 
access student progress. 

The most dramatic change was in the teachers’ attitudes about their role. Originally they felt 
responsible for the positive development of children but many saw their job as plagued with 
difficulties brought by the children. Many children spoke little or no English, were recent 
immigrants and unfamiliar with American educational practices and cultural ways, were poor 
and lacked the resources in their homes to support learning and positive children’s development. 
Family problems brought some children to school with high emotional needs that often resulted 
in classroom disruptions. The children, their families and home environments were described as 
“the problem” for children not learning, and seen by teachers as barriers they could not 
overcome. 

Over the next three years teachers’ attitudes shifted. They began to see themselves as having the 
ability to influence learning and the program gave them additional tools to use. Their “tool box” 
of instructional practices and techniques expanded over time, increasing the teacher’s comfort 
with their role. They moved fi'om blaming children and families to demonstrating leadership by 
looking for more and new ways to help children learn. Teachers became intentional in their 
practices. Classroom routines were established that reinforced literacy development. Teachers 
quickly realized that the routines reinforced literacy skills in children. The link between 
classroom practices and instructional techniques to children learning was formalized. Teachers 
challenged themselves and their mentors with finding additional ways to help children learn. 
Transitions to various classroom activities became opportunities to reinforce learning. For 
example, children began their day “signing-in” to their classroom. Children sat on shapes on the 
floor and lined up in alphabetical order. Rhymes were recited before going out to recess. Peas 
and carrots were coimted during limch. Songs about numbers and ABCs were simg on the way 
to the bathroom. Literacy skills were consistently integrated into activities throughout the Head 
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Start half-day. Teachers actualized their role of creating an environment and activities that 
stimulated children’s learning and positive development. What had been problems caused by 
others became opportunities for professional developed and increased student success. 

While changes in teacher attitudes helped teachers develop their intentionality, they also 
increased their skill with the SPARK curriculum. As teachers became familiar with the 
curriculum and modified it, they became more comfortable and natural in the classroom. At the 
end of the first year, teachers complained about how some of the stories in the SPARK 
curriculum were inappropriate for their children. They continued to use these activities even 
though children were disengaged. In the second year they modified the ineffective activities and 
added their own. As their knowledge of the curriculum grew so did their ability to modify the 
curriculum and create more effective practices. With more of the curriculum known and 
miderstood, teachers deviated from the standard and seized the teachable moments to respond to 
student characteristics and progress. Although only 40 percent of the original teaching staff 
remained at the beginning of the third year, even the new teachers adapted the curriculum 
quickly to address their children’s needs. Learning became part of the climate at the centers and 
mature and experienced teachers provided support for the new teachers with effective models 
and proven teaching activities and techniques. In addition, teachers’ knowledge of the stages of 
literacy development grew. In the third year, “the road to literacy development” that described 
children’s literacy developmental levels helped teachers create activities that developed a 
particular stage. The sequencing was established that allowed the teachers to provide the 
appropriate activity. Since little of the knowledge base had been previously developed, the 
teachers developed a knowledge through their experiences and tracking of student progress. 

They began to see the developmental stages and levels in their children and could identify new 
stages that had to be taught. The added knowledge enhanced their teaching ability/effectiveness. 

The children’s cultures and languages have shifted fi"om being strictly a barrier to learning to 
becoming an opportunity for learning. In three centers, all children learned to cornit and greet 
each other in all of the children’s languages, expanding their language base. Teachers 
encoiunged children to talk with one another even if it was not in the same language. Language 
became the centerpiece of the classroom. Parents of English-speaking children asked teachers 
for Spanish and Hmong words so that they could encourage the multiple language development 
at home. In contrast to early observations of English-speaking teachers turning away when a 
child spoke to them in their home language or sending them to someone who spoke then- 
language, teachers in two centers expanded their own ability to speak the children’s language. 
Teachers at the center with many Spanish-speaking children proudly proclaimed at the end of the 
year that almost all could understand Spanish. Teachers at another site learned to speak Hmong. 
They explained and evaluators observed that when teachers spoke to children in their home 
language children smiled. Children’s engagement in the classroom grew. Parents were 
pleasantly caught off guard when they heard the teacher speaking Hmong to the children. In 
addition, the role of the bilingual teacher changed. The Hmong-speaking teachers’ role changed 
fi-om one of a full-time translator constantly speaking Hmong to that of a classroom teacher and a 
bilingual resource only when the child needed the resource. Children who had some English 
ability were expected to speak English when speaking to the bilingual teacher. The rich 
language-based classrooms were noted by one special education teacher to be the result of 
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limited or non English-speaking children beginning to speak early in the year rather than at year 
end as in previous years. Case studies also illustrate parental satisfaction with bilingual teachers: 

When she has concerns, Lupe ’s mother speaks directly with her child’s teacher. 

Having a bilingual teacher is helpful to her. She also appreciated that the 
librarian speaks Spanish. What the mother valued most about her child attending 
Head Start is that the teacher is very good and helps her daughter a lot. She said 
that her “child demands a lot of patience and teacher and teachers have more 
patience than mothers. Teachers can get children to do things better than 
mothers. ” The mother credited her child’s teacher for helping to improve her 
child’s attitude and behavior. She is very satisfied with her child’s progress and 
feels that Lupe is well prepared. 

The program had numerous examples of how teachers integrated the children’s cultures into their 
classrooms by the end of the third year. Children’s cultures became part of the classroom, 
beginning with a few multicultural books and toys and expanding to the children’s home 
language being spoken by teachers and children, words in the all children’s languages labeling 
the classroom and family pictures on bulletin boards. Teachers grew in their ability to respond to 
varied cultural practices When Somali parents in the second and third years complained about 
classroom practices that did not respect their culture, teachers were defensive. In the third year, 
the defensiveness changed to problem-solving and finding resources to support Somali cultural 
practices. Dialogues began between Somali parents and teachers to learn about the Somali 
culture and the Head Start program and educational practices. The process of teachers and 
parents learning from one another has the potential be used for other immigrant groups as they 
arrive and become part of the Head Start community. 

While most teachers seized the program as an opportunity to expand their teaching skills, a few 
were stymied and stuck. Quality of teaching across classrooms was inconsistent. A few teachers, 
in spite of additional training and coaching, did not demonstrate the skills necessary to help their 
multicultural, multilingual children develop literacy skills. Literacy growth rates lagged in their 
classrooms. In addition, a few teachers demonstrated attitudes and behaviors that were not 
conducive for learning. For example, at least two were observed making ethnically biased 
statement about the children they teach. The greatest inequity of the program was the vast range 
in teaching abilities. Some children were taught by outstandingly effective teaehers and others 
received less competent instruction, thus jeopardizing those children’s preparation for 
kindergarten and thirst for learning. 



Annual Findings 
1999-2000 (First Year) 



Strengths: 

• Specific instructional techniques: Repetition, dialogic reading, and connecting SPARK 
themes to auxiliary activities supported literacy development. 

• Print-rich environment: Throughout this first year, classrooms became more “print-rich.” 
Labels, books, and writing activities were added to most classrooms. 
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• Multi-sensory, hands-on activities: Teachers early on seemed very adept at building these 
activities into their day and coordinating them with SPARKS activities. 

• Teacher flexibility: As teachers became more familiar with the Words Work! program and 
SPARK activities, they became adept at modifying activities to best serve their students. 

Challenges: 

• Teachers’ varying abilities: While most teachers excelled at finding ways to support 
children’s literacy development, there were varying levels of proficiency. For instance, 
evaluators noted that a teacher’s deliberate connection to literacy seemed to make the largest 
impact on the children, but they also noted that some teachers were more adept at making 
these connections than others. 

• Multi -cultural and -lingual activities and materials: Although evaluators noted an increased 
presence of multi-cultural books, toys and activities by the end of this first year, they also 
noted that these materials often remained scarce, especially in recent-immigrant languages 
like Hmong and Spanish. Parents at spring focus groups also asked for more materials in 
their home languages. 

• Access to computers: With one computer in each classroom, children were observed standing 
in line to wait for computer time. 



2000-01 (Second Year) 



Strengths: 

• Age-specific classrooms: Three sites hosted 3- and 5-year-old classrooms, which seemed to 
allow teachers to concentrate more on specific children’s needs. 

• Print-rich environment: In addition to labels, books, and writing activities, most teachers this 
year began having children “sign-in,” either by finding their names on cards or printing their 
names themselves. 

• Multi-sensory, hands-on activities: Teachers continued to develop and use creative multi- 
sensory instructional practices. 

• Teacher flexibility: Instruction continued to move from “one approach works well for all” to 
meeting individual student’s developmental needs. 

• Staff professionalism: Teachers and aids were increasingly able to create appropriate 
instruction and to use meeting time to plan and reflect purposefully. Teachers’ attitudes 
generally switched from defensiveness to problem solving. 

Challenges: 

• Teachers’ varying abilities: While observers reported that teachers made gains in areas such 
as intentionally coimecting instruction to literacy and engaging the children, teachers 
continued to display varying abilities to translate classroom materials and situations into 
learning opportunities for children. 

• Lack of bilingual staff: Although bilingual staff were present in most classrooms, some 
participants in parent focus groups reported dissatisfaction with a lack of bilingual staff in 
their children’s classroom. 

• Computer access: As the computer continued to be a popular learning tool, one computer per 
classroom continued to present a challenge, with children waiting in lines. 
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2001-02 (Third Year) 



Strengths: 

• Intentional practices: Teachers were consistently reported to be intentionally introducing 
literacy to the classroom environment. 

• Integrating SPARK concepts: Teachers were noted to have become much more comfortable 
integrating these concepts into their days by the third year of the program 

• Teacher flexibility: Teachers continued to find “teachable moments” and determine how to 
use a variety of strategies to meet individual children’s needs. 

• Teacher professionalism, development of teaching strategies: Teachers were observed having 
moved away from attitudes of “How are we going to fit this in?” and “These children can’t 
do this,” to being able to take advantage of every teaching opportunity. 

• Print-rich environment: Classrooms continues to evolve as writing and print-rich centers of 
learning. 

• Multi-sensory, hands-on activities: Teachers continued to develop and use creative multi- 
sensory instructional practices. 

• Bilingual staff: At several centers, staff were observed speaking children’s home language, 
even if the teachers were not very familiar with the language. 

Challenges: 

• Computers: Many parents continue to comment that they would like additional computers in 
their children’s classrooms. 

• Teachers’ varying abilities: Some teachers are still reportedly not performing at an acceptable 
level. 

• Discipline issues: Discipline issues continue to disrupt classrooms and teachers are often 
prevented from dealing firmly with disruptive students. 
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Family Involvement 



The program successfully increased parents’ involvement in their children’s education and 
family support for literacy development. The depth of the observations about program impact is 
limited because of minimal data. From the data collected in the parent and staff focus groups 
and home inventory comparisons, several observations can be made about the three-year 
progress. First, family involvement has increased, particularly in the area of at-home 
involvement. Parents reported that they have been using the teachers’ and home visitors’ 
suggestions to support their children’s literacy activities. For three years the Book-in-a-Bag 
program continued to be parents’ most used and favorite school project. For many families it 
provided the only children’s book in their home. A regular place for children’s school work has 
been established in most homes. Parents regularly asked their children about school work and 
reading with the child has increased considerably. While the home environment still lacks many 
of the resources (such as children’s books, games, blocks, and writing materials) parents began 
using home materials they do have for teaching their children. Labels on cans and boxes became 
reading materials. Signs were read while driving to identify letters. Parents reported their 
children practicing writing their names on any paper in the house. Parents attending the focus 
groups continued to request more activities they can do with their children to support their 
learning. Case studies reinforce the observation that parents are helping their children learn at 
home: 



Lupe lives with her mother, father and brother. Her family has been in the U.S. 
for four years, all of which have been in St. Paul. Her mother is currently a 
housewife. Her educational background is completion of a high school degree. 
This is Lupe ’s second year at Head Start. At home, Lupe likes to read books, tell 
stories, and works on puzzles. She also likes to help with cooking and setting the 
table. When helping to cook, she counts things. Toward the end of the school 
year, she would write her name, cut, draw, cook, pretend to read books and 
memorize books, but only in English. Activities that Lupe and her mother do 
together include reading, cooking, drawing, counting, playing outside, going for 
walks and visiting the children ’s museum. Her mother felt that reading, counting 
and talking about things helped her daughter learn. Reading is something that 
the family is able to do regularly. The mother, father, and older brother all help 
Lupe read. 

Fahima and her mother came from Somalia and have been in the U.S. for five 
years, all of which have been in St. Paul. Her father and siblings are still in 
Africa. Fahima ’s mother ’s reason for enrolling her in Head Start was that her 
daughter needed to learn more things and improve her English skills to prepare 
her for kindergarten. The mother does know her daughter ’s teachers. She said 
she mostly used the books and newsletters to help support her daughter ’s 
learning. What the mother valued most about her daughter attending Head Start 
is, “She has become better at expressing love to family. She obeys better and 
plays better with other children. She also reads better. ” The mother would like 
to see more computers available at Head Start and more work activities. 

However, she is very satisfied with her child’s overall experience. At home. 
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Fahima does the following activities: color, count, read stories, attend cultural 
activities in the Somali community, and other activities to learn about her culture. 
The mother does help her child with homework, but because she is now working 
full-time, it is difficult to do many things with her child. She does have some 
people in the Somali community who help her daughter with reading books. 



As children came home proudly demonstrating what they learned and communication from 
teachers increased, parents beeame increasingly aware of their child’s progress. The information 
was well received by the parents. The more parents knew the more they wanted to know. In the 
third year, parents in some sites asked for written comments from teachers once a week. The 
newsletters were effective ways the school communicated about classroom activities, themes and 
activities families could complete to support their child’s classroom work. Telephone calls, 
verbal reminders when parents picked up their children and bus driver comments were the most 
effective ways the program used to let parents know about upcoming events requiring their 
attendance or participation. Monthly parent meetings changed from the first and second year 
when committee reports were tediously translated into two or three languages, to the third year 
when meetings consisted of demonstrations and parent activities mimicking children’s activities. 
This change also helped to keep parents well informed. Parents also appreciated increased 
eontact with their child’s teacher at the monthly parent meetings. 

Lupe ’s mother stated that she had enrolled her daughter in Head Start because 
she felt her daughter needed to know people, other than her family. She felt that 
this would not only help improve her behavior, but also, it would prepare her for 
school and help her learn English. Her mother knew her teacher and had met 
with her. She talked about being able to speak in Spanish with her child’s 
teacher. She used everything that is sent home to help her child learn, including 
bilingual books, and activity sheets. Interestingly, the mother said that she liked 
the bilingual books, but her daughter preferred to be read to only in English 
Lupe ’s mother identified several activities that support her child’s learning. They 
included helping her to focus on writing, having a scheduled time for reading, 
and listening to the child when she wants to share her ideas. What is difficult for 
the mother is that she has difficulties reading in English, but her daughter prefers 
to be read to in English She is interested in obtaining a library card, but does 
not know how to do so. The evaluator explained the process to her. The mother 
said that there used to be a home visitor, but that no one has come for months. 

She liked having the home visitor come and missed that interaction. 



Parents with a few exceptions were overwhelmingly pleased with their child’s Head Start 
experience and development of literacy skills. For parents with limited English abilities, 
bilingual staff members helped them feel welcome and informed. However, at one center, 
parents painfiilly retold stories about their children being bullied, hit and verbally abused by 
other children, causing their child to not want to attend school. Serious disruptive behavior in 
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the classroom continued to plague some classrooms, putting learmng at a stand still and 
interfering with children’s positive school experience. 

Family involvement is the least developed part of the program and remains the greatest 
challenge. Family involvement, like effective teaching practices, varied across classrooms. 

Some teachers are adept at communicating with parents and involving families in creative ways. 
In those classrooms, family pictures grace the walls. Family cultural symbols are used as 
learning opportunities. Parents were warmly welcomed and meaningful activities were found for 
them to do in the classroom. Other classrooms minimally involved parents. Few examples of 
family life, culture or activities were encouraged. Frequently written materials went home that 
were not translated into languages the parents can read. In addition, the home visitor component 
of the program has only been operating for the past one-and-a-half years. The hiring practices of 
Head Start complicated and slowed the home-visiting component of the program. Roles of home 
visitors are still imclear to many families and teaching staff. Parents still confuse home visitors 
with family advocates and case study evaluator, all who visit families in their home. 

The Head Start’s enrollment practices also interfered with family involvement. Parents in the 
first and third years complained that slow admissions practices caused long waiting lists and late 
admission. Children entered classrooms where relationship among their classmates had already 
begun to solidify. Parents requested an afternoon or morning classroom because of their 
schedules and are imable to get it. In the early days of the school year, parents participate in an 
orientation practice that they report overwhelmed and confused them, particularly recent 
immigrants who are unfamil iar with American educational practices and government regulations. 

Factors such as parents’ finances and work schedules also influence family involvement. One 
case studies illustrates this: 

Neng is attending HOPE Academy charter school for the 2001-02 academic year. 

She is in first grade. As the youngest child, she receives help jrom her older 
siblings. Her father works first shift while her mother works the second shift. 

According to the mother, the family is in a very stressful situation with both 
parents working and hardly ever seeing each other. The mother feels that she has 
very little time to spend with her children. Since she works in the evening, she is 
not able to be with the children when they come home at night. She cannot attend 
conferences so her husband has to do so. She stated that her husband was 
satisfied with their daughter ’s progress in school. Her teacher thinks she is doing 
well, but is still quite shy. 
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Annual Findings 
1999-2000 (First Year) 



Strengths: 

• Book-in-the-bag program: Focus-group parents reported this as the most-used parent-school 
activity. 

• Parent-teacher conferences: Focus-group parents reported being pleased with the 
conferences. 

• Fieldtrips: Focus-group parents reported enjoying the opportunity for their children and 
themselves to go on fieldtrips, especially the Children’s Museiun trip. 

Challenges: 

• Language barriers: Many focus-group parents reported a desire for more materials to be 
translated into their home language. 

• Communication: Many parents requested more information about classroom activities and 
their children’s progress in the classroom than they received. Specifically, monthly center 
meetings tended to be dominated by reports from various committees, and did not include as 
much information on children’s progress or classroom curriculum as parents would have 
liked. 

• Cultural attitudes to learning: Families’ attitudes about school seemed to affect parent 
involvement. Some families seemed to believe that all learning should take place at school. 



2000-01 (Second Year) 



Strengths: 

• Communication: Focus-group parents reported feeling informed about their children’s 
progress. Unlike the first year, no parents asked for additional feedback about their children 
this spring. 

• Book-in-a-Bag program: For the second straight year, focus-group parents reported this as 
the most-used parent-school activity. 

• Center newsletter: reading the newsletter was reported as the second most-used activity, and 
some parents reported a special appreciation of the newsletter being translated into their 
home language. The newsletter seemed to help the parents feel informed about their child’s 
progress as well as used home-based learning activities. 

• Center meetings: Reflecting parents’ concerns from the previous year, this year center 
meetings focused more on children’s progress and classroom curriculum. 

• Teacher home visits: Focus-group parents mentioned these visits as effective in helping them 
learn about their child’s progress and how they could support their child’s learning. 

• Home Literacy Visitors: Four home literacy visitors visited with 100 families in an effort to 
identify individual parent needs and develop and share information and materials with 
parents. 

Challenges: 

• Literacy home environment: the majority of the 36 homes profiled by Home Visitors lacked 
many of the materials that would support a child’s literacy development, such as a regular 
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place for children to do school activities and adequate learning materials. Focus-group 
feedback supported these observations. 

• Volunteering: For the second year, few focus-group parents reported participating in 
volunteer opportunities. 



2001-02 (Third Year) 



Strengths: 

• Literacy home environment: This year more than 75 percent of the 40 families profiled by 
the Home Visitors had a regular place for their child to do homework and 100 percent had 
writing materials and children’s books in the home. 

• Parental involvement at home: More families reported doing various activities with their 
child this year, according to surveys by Home Visitors. Parents were especially active in 
looking through their child’s backpack and talking with their child about school. 

• Family involvement in reading: Families were reading more to their children this year and 
using a wide variety of materials, according to surveys by Home Visitors. Also, almost 60 
percent of home visit families reported having a library card in Spring 2002. 

• Family-child interaction: Home visit families reported working on a variety of literacy 
activities at home with their children, such as working on colors and writing names. Playtime 
was reported to have decreased form the previous year. 

• Awareness of classroom activities: Parents’ level of awareness of activities increased slightly 
from the previous year, according to surveys by Home Visitors. 

• Book-in-a-Bag: This activity continues to be popular among families. 

• Home Literacy visitors: These home visitors were repeatedly cited as providing valuable 
support to families. 

• Center newsletter: This continues to be a favorite form of communication, and at some sites 
it has increased in frequency and become multi-lingual (although at one site, it has ceased to 
be printed in languages other than English due to an increase in the number of languages 
spoken by families and a lack of resources). 

• Parental enthusiasm: Teachers at most sites reported parents being more enthusiastic this 
year, and some parents reported visiting classrooms more often. 

Challenges: 

• Parent focus group involvement: Spring focus group attendance was down from fall 
attendance at three sites this year (although it increased at one site). 

• Reaching all parents at their own level: The program has not demonstrated an ability to reach 
many parents who may have cultural or financial barriers to participating in their children’s 
learning experiences. 

• Parents’ desire for adult literacy and community resources: More parents are increasingly 
asking for resources and help for themselves rather than children-centered help. 

• Streamlining resources: Some parents feel there are too many different people visiting them 
at home (teachers, advocates, home literacy visitors, case study evaluators) and others are 
confused as to who is there to provide what service. 
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Sustainability 



One of the Words Work! goals was that the program would be sustained by the collaborating 
agencies after the foundation funding ended. Words Work! has made sizable gains in ensuring 
sustainability but sustainability is still questionable. Several factors contribute to the program’s 
continuance at Head Start, including the program’s staff members’ strong commitment to 
developing children’s literacy and their knowledge of effective instructional techniques. Many 
of the program’s teaching strategies and classroom materials have become part of the non-Words 
Work! classrooms. Recent interviews suggested that competition between program teachers and 
non- Words Work staff members has been reduced and non- Words Work staff members seem 
anxious to learn about the program and replicate the effective teaching strategies. MLC staff 
members have warmly welcomed the program’s home visitors and continued to link the program 
staff with adult literacy resources. 

In spite of the program’s grounding in staff members’ skills, enthusiasm and commitment, the 
challenges are formidable. Head Start’s structure, personnel policies, leadership and 
communication continue to interfere with successful program outcomes and institutionalizing 
Words Work!. The lack of an effective system of accountability is foremost. The inequity 
among classrooms and centers lessened with training and strong support but were not eliminated. 
Accountability is required. Low pay scale and pay raises based on tenure rather than merit 
remain disincentives for teachers to improve and stay at Head Start. Only 40 percent of the 
starting teachers remain in the program after three years. Little time exists for teachers to reflect, 
plan and modify instruction. For three years teachers continuously explained how planning time 
was filled with completing paperwork with little relevance to their classroom practices. The 
program was forced to pay teachers for training and planning time. Time-consuming hiring 
practices kept potential candidates from being selected and positions were frequently imfilled. 
The lack of communication among centers, agency divisions and management prevented all staff 
members from understanding the program, its impact and relationship to their job. Resistance 
grew when uncertainty existed. Plans for replication become tenuous when top leadership 
positions are vacant and interim staff people are making decisions. The lack of leadership and 
charges of institutional racism have created a strained atmosphere of tension and uncertainty. 

The integration of the program into the MLC structure is also challenging. The home visitor 
program is a new venture for MLC. Pre-school children and their families are new target groups. 
MLC continues to question the fit of the Words Work! program with the agency’s focus on adult 
and school-age children’s literacy. 



Annual Findings 
1999-01 (First Year) 



Strengths: 

• Staff enthusiasm: Words Work! staff reported being greatly challenged, but also 
demonstrated a desire to learn about best practices and improve their own teaching skills. 

• Teacher flexibility: Words Work! teachers demonstrated great ability to both adapt to the 
challenges of the program and to modify the curriculum to meet the special needs of 
individual students and their families. 



Challenges: 

• Staff pay: The low Head Start pay schedule made it difficult to recruit and retain staff. This 
leads to negative results including high turnover and little incentive for staff to excel. 

• Staff time: Staff members had little time for planning and reflection. In addition, the 
positions of center manager and teacher were combined at two of the sites, leading to many 
instances of children being shortchanged while a staff member was called out of classroom to 
perform administrative duties. 

• Communication: Little communication existed among the various organizational levels of 
head Start about Words Work! 

• Philosophical issues: Philosophical differences between the Words Work! program and the 
early childhood development field presented challenges to the program. The early childhood 
field has concentrated on social, physical and emotional growth rather than literacy skill 
development. Traditional classroom delivery of literacy skills were seen as not 
developmentally appropriate for pre-school children. 



2000-01 (Second Year) 



Strengths: 

• Identifiable program outcomes: Words Work! identified outcomes presented a model for 
Head Start programs undertaking an agency-wide effort to develop outcomes. 

• Strong administrative support: The Head Start associate director strongly advocated for the 
Words Work! program. 

• Staff influences: Words Work! staff was effective in helping train non-Words Work! Head 
Start staff in both formal and informal ways. 

• Piloting concept: Such Words Work! pilot ideas as the 3-year-old classroom were considered 
for integration into Head Start program 

• Professionalism: Words Work! introduced an increased level of professionalism into Head 
Start program. For instance, outcomes were developed, and planning and teamwork 
increased. 

• Staff enthusiasm: Words Work! staff members were observed to be taking on leadership and 
increasing risk-taking such as speaking up at meetings, creating curriculum and problem 
solving. 

Challenges: 

• Staff pay: A low and inequitable pay scale continued to hinder efforts to recruit and retain the 
best teachers. 

• Staff time: Lack of time for teachers to plan and reflect continued to present a challenge. 
Head Start paper work took limited time away from planning time. Additionally, teachers 
continued to hold responsibilities as center managers, stretching their availability beyond 
reason. 

• Discipline policies and procedures: Agency discipline regulations and minimal resources for 
seriously dismptive children cause some children to wait six to nine months to receive the 
appropriate services. In the mean time, children in the classroom with these children suffer 
and learning is hindered. 
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• Communication: a lack of communication among Words Work! divisions, such as family 
advocacy and education was observed. 

• Staff tensions: Competition between Words Work! and non- Words Work! staff members led 
to tension among many centers. 

• Management leadership: Head Start management generally lacked understanding of and 
support for Words Work! goals, contributing to tensions among the agency. 

• Lack of strategic plan: The lack of a Head Start strategic plan complicated the possibility of 
Words Work! becoming a part of the program’s long-term goals. 

2001 -02 (Third Y ear) 



Strengths: 

• New structural model: A new Head Start model with a Words Work! site at the core of each 
cluster was developed and considered as an option for restructuring. 

• New materials: Classroom books and literacy materials for all Head Start classrooms were 
purchased. 

• Staff training: Literacy-based instructional training for non- Words Work! staff members 
occurred. 

• Impact on Minnesota Literacy Council (MLC): MLC has begim exploring a relationship with 
young children when they have worked exclusively with adults and school-age children. 

• Words Work! leadership: In spite of tremendous agency chaos from agency’s leadership 
leaving and charges of institutional racism, the program’s coordinator kept staff members 
focused on children and learning. 

• Increased awareness: Requests for project lessons learned and questions about the 
relationship between Words Work! and the literacy department show an increased level of 
awareness of Words Work! program by Head Start staff. 

• Teacher enthusiasm and professionalism: Many teachers were observed sharing strategies, 
singling out weak areas to work on and modifying instruction based on assessment data. 

• Professional development: Staff is getting direct feedback and the opportunity to practice and 
share what works with each other. 

• Program goals: When centers and classrooms had specific goals and focused on them, 
positive results on learning are seen. For instance, one center’s emphasis on increasing 
rhyming skills led to leaps in children’s rhyming performance. 

Challenges: 

• Institutional climate: Tremendous change among leadership occurred, with the directors of 
Head Start and Ramsey Action Program leaving. In addition, a charge of institutional racism 
was investigated during much of the year. In addition, the smallest progress was made in 
reducing racism in classrooms and decreasing competition among Head Start centers. 

• Discipline: Teachers continue to be ill-equipped (on an institutional level) to deal with many 
behavior issues. 

• Accountability: Currently, no system exists for monitoring and ensuring the quality 
performance of all staff. Some sort of staff accountability imbedded into the system would 
help. 

• Staff pay: This continues to be low. 



Staff time; More time and money for intense training would likely help teachers he 
prepared to meet program challenges. 



CRITICAL FACTORS 



Over the three years, several factors consistently influenced the success of the program. 
Children’s literacy abilities grew due to the curriculum and instructional practices, in particular: 

• The SPARK curriculum; 

• The creation of children’s activities to reinforce the SPARK concepts, learn the ABCs and 
develop phonological skills and vocabulary; 

• Language-rich environment, including the home language of all children ; 

• Literacy-based routines that became a regular part of the child’s school day; 

• Intentionally focusing on the desired literacy skills; 

• Bilingual teachers to engage all children; and 

• Integrating literacy activities into the all parts of the school day. 

Specific teaching techniques and strategies for each literacy outcome are listed in the appendix of 
this report. 

The training, mentors and time to reflect and share effective instructional practices were effective 
ways Words Work! improved teaching abilities and instructional practice. 

Family involvement and support for their child’s literacy development also grew. Several 
program activities and practices encoiuaged family involvement, including: 

• The literacy home visitors linking family activities with classroom activities and providing 
children’s books and other literacy materials for family reading; 

• The Book-in-the-Bag program allowed children to bring home a children’s book ,oflen the 
only or one of a few children’s books in the home; 

• Center and classroom newsletters informing parents about classroom activities and ways 
parents could support their child’s continued learning; 

• Monthly center meetings that simulated the child’s classroom experience and increased 
contact with the child’s teacher; 

• Parent-teacher conferences; 

• Bilingual staff members and classroom practices that respected family culture; and 

• Verbal commimication by teachers or bus drivers about events and child’s progress. 

Insufficient data exist at this time to determine if family involvement activities spurred children’s 
literacy development. 

The sustainability of the Words Work! program was also enhanced by several critical factors, 
such as: 

• The strong leadership of the program coordinator and involvement in the Head Start 
management decisions; 

• Training of non-Words Work staff members in literacy-based instructional techniques; 
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• Shared information about the Words Work! philosophy, values, assumptions, goals and 
impacts with Head Start management and non-Words Work! staff members; and 

• National Head Start’s requirement to develop outcomes, including literacy and evaluating the 
local program’s effectiveness of achieving the outcomes. 

While the program brought about many desired changes and progressed toward program 

outcomes, several challenges exist. Many of these challenges persisted over the three years, 

including: 

• Institutional barriers that interfere with children’s learning, family involvement. Head Start 
communication, accoimtability, leadership and program replication; 

• Limited data and evaluation of the home-visiting component of the program and its 
relationship to children’s literacy skills; 

• Minimal strategies to involve families of varying cultures; 

• Sharing of effective instructional practices and development of teaching skills, particularly to 
build children’s phonological skills, ABCs and conceptual vocabulary; 

• Wide range of teaching abilities across classrooms and centers; and 

• Sustaining the costly home visiting component within an agency that has no previous 
experience with pre-school children and their families. 
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RECOMMENDATIONS 



The Words Work! program has been particularly adept at addressing recommendations included 
in each year’s evaluation report. Program leadership and center staff members have been quick 
to solve problems and find creative solutions. Unfortunately some challenges still remain. The 
recommendations below are offered to improve children’s literacy development, instructional 
practice, family involvement and replicate and sustain the program beyond foundation funding. 
Specific suggestions include; 

• Continue and enhance the same pattern of staff development that includes training, direct 
feedback, mentoring, videotaping, reflection and sharing of best practices; 

• Clarify teacher expectations of performance and develop a system of accountability to ensure 
quality performance of all teachers, in particular, target areas for improvement with follow- 
up; 

• Increase teacher understanding of phonics and its relationship to literacy and reading and 
continue to develop and share effective teaching strategies for phonological skills, 
particularly rhyming, vocabulary and ABCs; 

• Set program standards that outline pre-school literacy skills for children’s success in 
kindergarten; 

• Improve the discipline policies and system, using classroom teacher ideas, to reduce the 
classroom disruption due to children with severe behavior problems; 

• Increase coordination among the home visitor, classroom teachers and family advocates; 

• Continue to develop unique strategies to involve families of varying cultures, including 
increased communication in the home language and culturally-appropriate materials; 

• Ensure bilingual teachers with the same languages as the children; 

• Investigate the relationship between family literacy practices and children’s literacy 
development; 

• Replicate best practices and continue to train staff members in non-Words Work! sites; 

• Involve teachers in enrollment decisions; and 

• Develop a strategy to inform and involve new Head Start leaders in developing policies and 
procedures that support the program’s sustainability and children’s literacy development. 
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CONCLUSION 



The Words Work! program has progressed toward achieving its outcomes over the past three 
years. In particular, the program has successfully helped pre-school children master many 
literacy skills, including knowledge of a book’s operation, meaningful covmting, ability to write 
their name and vocabulary including colors, opposites and many other language skills. While 
growth in phonological sldlls, conceptual vocabulary and ABCs was not as great as other literacy 
skills, the growth was sizable and statistically significant. In addition, the growth for children of 
color accelerated and reversed the two-year trend of parallel progress. The long-standing 
achievement gap between children of color and white children began to close this year. 

The accelerated growth and mastery is due to the effective instructional practices of program 
teachers and a literacy-dominated curriculum. Teachers continue to develop new and creative 
ways to teacher children literacy skills. Unfortunately, teaching abilities vary fi'om classroom to 
classroom and are reflected in differences in children’s mastery of literacy skills. 

Family involvement also grew over the three years. The first year of the home-visiting 
component of the program completed this year, resulting in increased family involvement and 
support of their child’s literacy development. Home reading increased, the home learning 
environment of participating families became more eonducive to literacy development and 
parents’ knowledge of classroom activities increased. Family enthusiasm for the program 
continued to grow, with families often in focus groups offering few suggestions. 

Challenges remain. Significant institutional barriers continue to interfere with children’s literacy 
development, family involvement and the sustainability of the program. These barriers must be 
addressed. TTie program must continue to be strengthened in the existing centers, particularly in 
areas of phonological skill, conceptual vocabulary, ABCs and family involvement. At the same, 
incremental replication of Words Work! into non-program sites should continue. 
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APPENDIX 



CRITICAL FACTORS 
Instructional Techniques 



1999 - 2000 

• Deliberate connection between learning and classroom activities stimulated literacy 
development. 

• Multi-lingual education improved effectiveness of instructional techniques. 

• Socialization provided critical learning behaviors for school success. 

2000 - 01 

• Deliberate connection between learning and activities continues to stimulate learning. 

• Creation of age-specific classrooms led to improvements in behavior and social skills 
observed by teachers. Assessment results could not confirm these observations. 

2001-02 

• Intentionality: Teachers have continued deliberate connection between learning and 
activities. 

• Praising children for their efforts has reinforced positive learning associations for them. 

• Incremental learning: Many teachers have begun to break down the learning process into 
incremental steps, which helps children learn. 

• Multi-lingual classrooms: using more languages in the classroom helped boost children’s 
self-esteem and fostered the learning process. 

Staff Development and Support 

1999 - 2000 

• Teachers assumed responsibilities and challenges of fairly rigid SPARK curriculum with 
enthusiasm. 

• Staff used evaluation results to improve programming. 

• Staff modified curriculum to address special needs of individual students. 

• Extra day of staff development for next year was scheduled, without reducing classroom 
hours. 

2000 - 01 

• Experience from first year contributed to teachers’ ability to modify curriculum to meet 
center goals. 

• Teachers benefited from additional support from literacy staff, mentors, home visitors, family 
advocates, and bilingual staff members. 

• Teachers did not have necessary support to meet the special needs of very disruptive 
children. 
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2001-02 

• Mentor teachers in the classrooms have demonstrated techmques and initiated more 
strategizing. 

• Flexibility; Teachers have learned to use a variety of strategies in order to reach individual 
children. 

• Staff have learned to speak languages other than their own. 

• Teachers have continued to modify instruction based on evaluation data. 



Parent Involvement and Family Literacy 

1999-2000 

• Negotiations were begun with Minnesota Literacy Council to oversee family literacy 
outreach workers. 

• Head Start famil y advocates became more integrated into Words Work! program. 

• Parent attitudes about school and teaching need to be sensitively addressed. Increasing 
parents’ understanding of Head Start and Words Work! will affect these attitudes as well as 
meet parents’ requests for more information about their children’s progress. 

2001-02 

• Home literacy visitors have added an important connection between families and school. 

• Oral announcements for events have more impact in getting parents to attend. 



Culture 

1999 - 2000 

• Staff understanding of cultural issues will impact both children’s learmng and parent 
involvement. 

2000 - 01 

• Staff understanding of cultural issues will impact both children’s learning and parent 
involvement. 

2000-01 

• Staff understanding of cultural issues will impact both children’s learning and parent 
involvement. 



Institutional Practices 

1999-2000 

• Having combined position of center manager and teacher at two sites shortchanged both 
pKJsitions. 

• Staff members never seem to have enough time to do all that is required and — many feel they 
have admini strative requirements that do not help children learn. 

• Staff turnover disrupts children’s learning. 
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• RAP procurement and hiring process is cumbersome and slow. 

• Low Head Start pay schedule makes it difficult to attract and retain quality staff. 

• Communication is poor among various organizational levels of Head Start about Words 
Work!. 

2000-01 

• Lack of time for teachers to plan and reflect. 

• Minimal support or resources for staff to deal with very disruptive children. 

• Limited teacher consultation in classroom placement of children. 

• Teachers continuing as center managers. 

• Lack of communication among Head Start divisions. 

• Low Head Start pay schedule makes it difficult to attract and retain quality staff. 

• Inequitable pay system. 

• Competition among Words Work! and non-Words Work! staff. 

• Lack of management leadership in supporting Words Work! goals. 

• Lack of Head Start strategic plan of which Words Work! could become a part, 

• Less time for Words Work! project coordinator for project implementation and academics 
with her time increasingly spent at Head Start meetings communicating and strategizing to 
increase the likelihood of project sustainability. 

2001-02 

• New structural model: A new Head Start model with a WW! site at the core of each cluster 
was developed. 

• New materials: There was an increase in the purchase of classroom books and literacy 
materials for all Head Start classrooms. 

• Lack of time for teachers to plan and reflect. 

• Minimal support or resources for staff to deal with very disruptive children. 

• Limited teacher consultation in classroom placement of children. 

• Program goals: When centers and classrooms have specific goals and focus on them, positive 
results on learning are seen. For instance, one center’s emphasis on increasing rhyming skills 
led to children’s increased rhyming performance. 
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RECOMMENDATIONS 



Effective Instructional Techniques 



2000 

• Continue to train staff members in new instructional techniques and find ways to share 
effective practices among all staff members. 

• Find ways to supplement SPARK curriculum with materials and activities that develop 
phonological awareness, vocabulary, ABCs, and counting. 

• Continue to use wall materials to support instruction. 

• Find more equitable ways for all students to use computers and link use direetly to literacy 
development. 

• Ensure that every classroom has bilingual staff members and continue to reinforce key words 
in children’s home languages. 

• Continue to develop children’s social skills and reinforce among staff members the 
importance of socialization to literacy and school readiness. 

2001 

• Provide additional classroom resources such as SPARK stories on tape and additional 
literacy materials besides books. 

• Provide additional bilingual teachers and staff members. 

• Assess children’s knowledge of ABCs in the fall to ensure accurate baseline data. 

• Set program standards that outline pre-school literacy skills required for children’s success in 
kindergarten. 

• Further examine the impact on literacy development of classroom factors such as staff 
turnover, instructional techniques, and age-specific classrooms. 

2002 

• Focus on vocabulary development and integrating vocabulary into all parts of the day. 

• Focus on the sequence of the learning process: breaking down the learning process into 
incremental steps. 

• Increase use of videos. 



Staff Development and Support 



2000 

• Find ways to support the professional development of all staff members. 

• Use meaningful ways to recognize staff effectiveness. 

• Provide resources and materials the staff requests to facilitate their teaching. 

2001 

• Increase time for reflection, feedback, and dialogue with teachers within and across centers. 

• Begin Fall vvith a review of the learning from Spring evaluation. 

• Provide adequate resources to address special behavior management. 
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• Challenge teachers to improve their instructional skills and move children to another level. 

2002 

• Continue with and enhance same patterns for professional development for staff: direct 
feedback, opportunity to practice, training, sharing. 

• Formalize best practices techniques. 

• Find time and money for more intense staff training, perhaps begirming-of-year training or 
summer sessions. 

• Focus teacher training on phonics and vocabulary development in order to increase teacher 
understanding of concept. 

• Develop a system to monitor and ensure the quality performance of all teachers and target 
areas that need improvement with effective follow-up techniques. 

• Improve both policy and system for teachers dealing with behavior problems. 

• Increase teacher input into classroom placement. 

Parent Involvement and Family Literacy 

2000 

• Increase reports parents receive about their children’s social and literacy progress. 

• Offer parents specific methods for supporting their children’s literacy development at home. 

• Spend most parent meeting time demonstrating classroom literacy activities and teaching 
techniques. 

• Find appropriate ways to explain to parents the various ways that children learn, the 
relationship of learning to developmental stages, and the Head Start approach to learning. 

2001 

• Find additional ways to involve families. 

• Coordinate the home literacy visitor vdth classroom work and other Head Start departments. 

• Continue to provide home learning activities for children and parents. 

2002 

• Work on master plan for outcomes of increased parent involvement. 

• Find ways to reach all families at their level. 

• Develop a more coordinated system of resources involving home literacy visitors, family 
advocates, evaluators, etc. 

• Find cost-effective ways of translating more newsletters and other materials into parents’ 
home languages. 



Institutional Practices 

2000 

• Eliminate combined position of center manager and classroom teacher. 

• Reduce administrative duties of instructional staff. 

• Facilitate the transition and training of new staff members. 

• Increase communication about Words Work! among various levels of Head Start and RAP. 

• Work with Head Start and RAP to find creative ways to speed hiring and procurement and to 
raise wages. 
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• Begin to discuss and address the philosophical differences between Words Work! and early 
childhood development field. 

2001 

• Eliminate combined position of center manager and classroom teacher. 

• Eliminate the competition between Words Work! and non- Words Work! staff members. 

• Consider educational needs of children in enrollment practices. 

• Address pay inequities of staff members. 

• Reduce time positions are unfilled. 

• Continue to communicate with and educate Head Start leadership about the project. 

• Continue to foster communication among Head Start departments. 

2002 

• Allow non-Words Work! sites to begin incrementally changing towards WW! program now. 




75 



44 



Words Work! Initiative 

Student Assessment Data — Fall 2001/Spring 2002 
Prepared by ACET, Inc. 

May 23, 2002 

Words Work! Head Start centers (Baker, Bigelow, McDonough, and St. Clair) used a variety of tests to 
measure student academic performance, to include the SPARK checklist, Henderson Phonological 
Assessments, ABC/Attempt to Print, and the Boehm Test of Basic Concepts. Alt the assessments were 
administered for each center in fall of 2001 and again in the spring of 2002. Only children who took both 
assessments in the fall and again the spring were included for longitudinal analysis. At St. Clair 48 of 5 1 
children (94 percent) originally tested in the fall took the assessments again in the spring. Fifty-five of 68 
children (81 percent) at Baker took the assessments in both periods. Bigelow and McDonough shared 
identical mobility rates at 79 percent. Following are the results from the assessments. 

SPARK Assessment 

Overall, all four Head Start centers showed an increase in the percent of Words Work! children who could 
complete each of the 40 activities on the SPARK Assessment'. 

• The largest range of growth rate was observed at McDonough (0 to 100 percent). 

• Baker had a growth rate that ranged from 4 to 100 percent. 

• The cohort group at Bigelow showed a growth rate that ranged from 26 to 93 percent. 

• At St Clair, the growth rate for children ranged from -4 to 89 percent. 



The table below presents activities with the largest proportion of student growth. 



Baker 


Bigelow 


McDonough 


St Clair 


AUCenters 


4+ body parts 
▲ 100% 
Conversations 
▲ 100% 
Points to red 
▲ 88% 
Names orange 
▲ 86% 
Names items 
▲ 85% 


Points to yellow 

▲ 93% 
Names Circle 

▲ 91% 
Name in print 

▲ 91% 

Short song/rhyme 

▲ 86% 
Names yellow 

▲ 85% 


Names circle 
A 100% 
Name in print 
A 100% 
Conversations 
A 100% 
2-step directions 
A 100% 

4+ body parts 
A 96% 


2-Step directions 
A 89% 
Name in print 
A 82% 
Points to red 
A 77% 
Conversations 
A 71% 

4+ body parts 
A 69% 


Name in print 
A 87% 

4+ body parts 
A 86% 
Names circle 
A 86% 
Conversations 
A 85% 
Points to red 
A 82% 



The table below shows activities with the smallest proportion of student growth. 



Baker 


Bigelow 


McDonough 


$t Clair 


All Centers 


Address 
A 4% 
Sequence 
A 31% 
Asks what if 
A 36% 

Names rectangle 
A 46% 
Names square 
A 47% 


Address 
A 26% 
Asks what if 
A 39% 
Opposites 
A 47% 
Own feelings 
A 48% 

Names rectangle 
A 49% 


Address 
A 0% 

Positional words 
A 28% 

Own feelings 
A 39% 
Sequence 
A 39% 

Short song/iiiyme 
A 43% 


Address 
T 4% 

Names rectangle 
A 26% 
Opposites 
A 33% 

Points to orange 
A 33% 

Names triangle 
A 37% 


Address 
A 9% 

Positional words 
A 45% 
Sequence 
A 46% 
Opposites 
A 47% 
Asks what if 
A 47% 



' To measure student progress, a conditional proportion formula was applied to calculate the proportion of chilcfren 
at each center that had room to grow, and did grow from fall 200 1 to spring 2002 (i.e. the percent of children eligible 
to grow in the fall minus the percent of children eligible in the spring divided by fall eligibility). Growth was 
defined as movement fiom “no” or “emerging” to “yes” on any SPARK activities. 






f u 



§EST S8PY AVAILABLE 



45 



Henderson Phonological Assessment 



Overall, Words Work! children on average made statistically significant progress from fall 2001 to spring 
2002 in all six areas of the Henderson Phonological Assessmenr and in the ABCs. The following table 
shows areas with significant differences in mean scores for each center followed by all centers (as 
measured by a T-Test). 



Center 


Area 

(Language) 


Possible 

Score 


Mean 

(Fall) 


Mean 

(Spring) 


Mean 

Difference 


Change 


Baker 


Book (Eng) 


(0-7) 


2.62 


5.43 


2.81 **(n=53) 


A 


Story (Eng) 


(0-4) 


1.09 


2.04 


0.94** (n=53) 


A 


Word (Eng) 


(0-6) 


2.00 


3.44 


1.44 ** (n=53) 


A 


Syllable (Eng) 


(0-6) 


1.43 


3.65 


2.22** (n=49) 


A 


Sounds (Eng) 


(0-6) 


0.49 


2.76 


2.27** (n=49) 


A 


Book (Home) 


(0-7) 


3.94 


5.76 


1.82** (n=l 7) 


A 


Syllable (Home) 


(0-6) 


1.25 


3.69 


2.44** (n=16) 


A 


ABC 


(0-26) 


3.34 


11.66 


8.31** (n=35) 


A 


Attempt to Print 


(1-6) 


3.98 


5.17 


1.19** (n=54) 


A 


Bigelow 


Book (Eng) 


(0-7) 


4.73 


6.00 


1.27**(n=101) 


A 


Stoiy (Eng) 


(0-4) 


1.79 


2.32 


0.52** (n=101) 


A 


Word (Eng) 


(0-6) 


2.30 


3.83 


1.53**(n=101) 


A 


Rhyming (Eng) 


(0-6) 


0.38 


1.59 


1.22**(n=101) 


A 


Syllable (Eng) 


(0-6) 


2.00 


4.08 


2.08** (n=101) 


A 


Sounds (Eng) 


(0-6) n 


0.21 


0.47 


0.26* (n=101) ^ 


A 


Book (Home) 


(0-7)^ 


5.44 


6.88 


1.44**(n=17)n 


A 


Stoiy (Home) 


(0-4) 


1.75 


2.31 


0.56* (n=15) 


A 


Word (Home) 


(0-6) n 


1.38 


2.25 


0.88* (n=15) 


A 


Syllable (Home) 


(0-6) 


0.94 ^ 


2.13 


1.19* (n=l 5) 


A 


ABC 


(0-26) 


6.41 


16.04 


9.63** (n=53) 


A 


McDonough 


Book (Eng) 


(0-7) 


5.20 


6.51 


2.90** (n=75) 


A 


Story (Eng) 


(0-4) 


1.69 


2.51 


0.96** (n=75) 


A 


Word (Eng) 


(0-6) 


0.67 


1.63 


0.93** (n=75) 


A 


Rhyming (Eng) 


(0-7) 


0.09 


0.52 


0.43** (n=75) 


A 


Syllable (Eng) 


(0-6) 


1.29 


4.17 


2.88** (n=75) 


A 


Sounds (Eng) 


(0-6) 


0.24 


1.05 


0.81** (n=75) 


A 


Book (Home) 


(0-7) 


5.12 


6.38 


1.27** (n=26) 


A 


Story (Home) 


(0-4) 


2.04 


2.81 


0.77** (n=26) 


A 


Syllable (Home) 


(0-6) 


0.65 


1.73 


1.08** (n=26) 


A 


Word (Home) 


(0-6) 


1.50 


2.92 


0.96** (n=26) 


A 


ABC 


(0-26) 


6.98 


15.32 


8.34** (n=44) 


A 


Attempt to Print 


(1-6) 


4.31 


5.55 


1.24** (n=75) 


A 



^ Children fluent in English were assessed in English, and children limited in English were assessed in both English 
and their home language. 
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Center 


Area 

(Language) 


Possible 

Score 


Mean 

(FaU) 


Mean 

(Spring) 


Mean 

Difference 


Change 


St. Clair 


Book (Eng) 


(0-7) 


5.11 


6.35 


1.24** (n=46) 


A 


Story (Eng) 


(0-4) 


2.22 


2.74 


0.52** (n=46) 


A 


Rhyming (Eng) 


(0-6) 


0.30 


1.39 


1.09** (n=46) 


A 


Syllable (Eng) 


(0-6) 


1.26 


3.07 


1.80** (n=46) 


A 


Word (Home) 


(0-6) 


0.57 


2.71 


2.14* (n=7) 


A 


ABC 


(0-26) 


4.40 


12.00 


7.60** (n=25) 


A 


Attempt to Print 


(1-6) 


3.47 


4.57 


l.ll**(n=47) 


A 


All Centers 


Book (Eng) 


(0-7) 


4.52 


6.09 


1.74**(n=275) 


A 


Story (Eng) 


(0-4) 


1.70 


2.39 


0.68** (n=275) 


A 


Word (Eng) 


iO-6) 


1.78 


2.95 


1.17**(n=272) 


A 


Rhyming (Eng) 


(0-7) 


0.24 


1.07 


0.82** (n=271) 


A 


Syllable (Eng) 


(0-6) 


1.58 


3.86 


2.28** (n=271) 


A 


Sounds (Eng) 


(0-6) 


0.26 


0.96 


0.70** (n=271) 


A 


Book (Home) 


(0-7) 


4.76 


6.23 


1.47** (n=66) 


A 


Story (Home) 


(0-4) 


1.80 


2.52 


0.71** (n=66) 


A 


Word (Home) 


(0-6) 


1.58 


2.74 


1.15** (n=65) 


A 


Syllable (Home) 


(0-6) 


1.00 


2.38 


1.38** (n=65) 


A 


ABC 


(0-26) 


5.57 


14.23 


8.66**(n=158) 


A 



* Significant at 0.05 ** Significant at 0.01 



Boehm Test of Basic Concepts 

Overall, Words Work! children on average scored statistically higher on the Boehm Test of Basic 
Concepts from fall of 2001 to spring of 2002. The possible range of scores is fi-om 0 to 52. The table 
below shows the mean difference for each center followed by all centers. 



Center 


(Language) 


Mean 

(Fall) 


Mean 

(Spring) 


Mean Difference 


Change 


Baker 


(Home) 


33.44 


39.06 


5.61** (n=18) 


A 


Bigelow 


(Home) 


25.06 


38.00 


12.94** (n=l 5) 


A 


McDonough 


(English) 


18.13 


28.92 


10.79** (n=75) 


A 


(Home) 


30.35 


37.62 


7.27** (n=26) 


A 


St Clair 


(English) 


27.09 


35.22 


8.13** (n=46) 


A 


(Home) 


19.56 


33.11 


13.56* (n=9) 


A 


All Centers 


(Home) 


28.52 


37.49 


8.97** (n=69) 


A 



* Significant at 0.05 
** Significant at 0.01 
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Factors that Influenced Test Scores 



All Centers^ 

A number of factors appeared to influence group test scores such as the children’s race, fluency in English, age, 

gender, enrollment status (i.e. whether the child is a new or returning student), and special needs status. 

o Race: Non-children of color tend to score higher than students of color on many sections of the 

phonological assessment and on the Boehm test. However, children of color have exhibited growth at a faster 
pace from fell-to-spring than non-children of color. 

o Fluency: Children who were fluent in English scored higher than non-fluent children on both the 

phonological assessment and the Boehm test. In addition, the learning curve of children fluent in English was 
slightly higher than non-fluent children on the phonological assessment and the Boehm test. 

o Age: Five-year olds speared to score higher on most sections of the phonological assessment 

and show growth from fall-to-spring at a faster pace with a higher learning curve than younger children. 

o Gender: Females tend to score higher than male children on one section of the phonological 

assessment and on the ABCs. 

o Enrollment: Children who attended Words Woric! during the 2000/2001 school year were 

scoring higher than newly enrolled children on many sections of the phonological assessment and on the Boehm 
test. 

o Special Education: Special needs status had a slight effect, with special needs children scoring 

higher than non-special needs children on one of the phonological measures. 

Baker 

The primary factors influencing group test scores include fluency in English, age, gender, enrollment status, and 

home visit status (i.e. whether or not the child received a home visit). 

® Fluency: Children fluent in English tend to score higher than non-fluent children on many sections of the 
phonological assessment and on the Boehm test. 

® Age: Five-year olds tend to score higher than younger children on many sections of the phonological 
assessment and on Attempt to Print 

® Gender: Males tend to score higher than female children on one section of the phonological assessment. 

® Enrollment: Children who attended Words Work! during the 2000/2001 school year tend to score higher on 

the Attempt to Print. 

® Home Visit: On one section of the phonological assessment, children who received a home visit tend to score 
lower than those children who did not receive a home visit. 



^ The degree the above factors influenced test scores vary with each center. 
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Bigelow 

A variety of factors appeared to influence group test scores, including race, fluency in English, age, gender, 

enrollment status, home visit status, and being in an age specific classroom. 

® Ra^e: Non-children of color tend to score higher than children of color on one section of the phonological 
assessment and on the Boehm test. However, children of color did score higher on another section of the 
phonological assessment. 

® Fluency: Children fluent in English tend to score higher than non-fluent children on many sections of the 
phonological assessment and on the Boehm test. 

® Gender: Males tend to score higher than female children on the Boehm test. 

® Age: Five-year olds tend to score higher than younger children on many sections of the phonological 

assessment. 

® Enrollment: Children who attended Words Worid during the 2000/2001 school year tend to score higher than 
newly enrolled children on one section of the phonological assessment and on the Boehm test. 

® Home Visit: Children who received a home visit scored higher on the Boehm test. 

® Age-Specific: Children enrolled in an age-specific classroom tend to score higher on the Boehm test. 

McDonough 

The primary factors influencing group test scores include race, fluency in English, age, gender, enrollment status, 

and special needs status. 

® Race: Non-children of color tend to score higher than children of color on some sections of the phonological 
assessment and on the Boehm test. 

® Fluency: Children fluent in English tend to score higher than non-fluent children on most sections of the 
phonological assessment and on the Boehm test. 

® Age: Five-year olds tend to score higher than younger children on many sections of the phonological 
assessment and on Attempt to Print. 

® Gender: Females tend to score higher than male children on one section of the phonological assessment. 

® Enrollment: Children who attended Words Worid during the 2000/2001 speared to score higher than newly 
enrolled children on two sections of the phonological assessment and on the Boehm test. 

® Special Education: Special needs children scored higher than non-special needs children on one section of the 
phonological assessment. 

St Clair 

The primary factors influencing group test scores include race, fluency in English, age, enrollment status, and 

special needs status. 

® Race: Non-children of color tend to score higher than cildren of color on two sections of the 
phonological assessment. 

® Fluency: Children fluent in English tend to score higher dian non-fluent children on most sections of the 
phonological assessment and on the Boehm test. 

® Age: Five-year olds tend to score higher than younger children on many sections of the 
phonological assessment and on Attempt to Print. 

® Enrollment: Children who attended Words Worid during the 2000/2001 school year, tend to score higher than 
newly enrolled children on most sections of the phonological assessment, the Boehm test, and on Attempt to 
Print. 

® Special Education: Special needs children tend to score higher on one section of the 
phonological assessment and on another section non-special needs children scored higher. 
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Words Work! Spring 2002 Parent Focus Group Summary June 12, 2002 



Introduction 

This report summarizes findings from parent focus groups held in April of 2002 at the four 
Words Work! sites. The purpose of the focus groups was to leam from parents about three areas: 

• Aspects of the Words Work! project that contribute to or impede children’s ability to begin 
reading, loving books, and learning to write and count; 

• Factors in the home that influence this progress; and 

• Suggested ways that the program can involve parents in their children’s learning. 

This report presents general themes mentioned by several parents at more than one Words Work! 
site. It is based on four site-specific focus group reports. 

Parent Demographics 

Eighty-four parents participated in the Spring focus groups. Almost 50 percent of the 
participants were at one site, and two sites had 1 3 or fewer parents attend the focus groups. Most 
of the parents were mothers, but many fathers also participated. Most reported being aware of 
and involved in their child’s classroom activities. Some were the parents of more than one Head 
Start child, or had children who had been in the program for two or more years. More parents 
were Latino than any other race or ethnicity. Most of the Latino-identified parents were 
Mexican, although some were fi-om other Central American countries. Most of the African 
parents were Somali. The following table shows self-identified race and/or ethnicity of the 
participating parents. 



Table 1: Race/Ethnicity of Parents 



Race/ 


Gender 


Site 


Ethnicity 




Bigelow 


Bal 


ker 


McDonough 


St. Clair 


All 


sites 


African 


Male 




7 




4 


1 


2 




0 


1 


13 




Female 


7 




4 




1 








6 


15.5 
























% 


African- 


Male 


1 


2 




2 




1 




0 


1 


5 


American 


Female 


1 




2 




1 








4 


6% 


American 


Male 




0 




1 




0 




0 


0 


1 


Indian 


Female 






1 












1 


1% 


Caucasian 


Male 




8 




0 




2 


1 


8 


1 


18 




Female 


8 








2 




7 




17 


21% 


Hmong 


Male 


2 


2 


2 


4 


2 


6 




1 


6 


13 


Female 






2 




4 




1 




7 


15.5 
























% 


Latino 


Male 


1 


3 


5 


26 




2 


1 


3 


7 


34 




Female 


2 




21 




2 




2 




27 


41% 


Site T 


otal 


22 


37 


13 


12 


84 


Percent of Total 


26% 


44% 


16% 


14% 


100% 
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Focus groups were conducted in English, Hmong, Somali, Spamsh, and Tigrigna. 



Findings 

Parent Perceptions of Children’s Learning 

As at previous focus groups, parents were generally pleased with their children’s progress in the 
Words Work! program. Many said that their children had learned a great deal. They also seemed 
as interested in their children learning social skills as literacy skills. Many commented on the 
positive behaviors their children have learned in Head Start. Several also mentioned the value of 
the program’s multi-cultural environment. 

Parents at all sites reported being most impressed with Words Work! teachers. They talked about 
how teachers are the key to the learning experience, how they show interest in each child, and 
how comfortable they as parents feel talking to their children’s teachers. A typical comment was: 

The attitudes of the teachers are great. They really want the 
child to learn. 

However, not all parents reported such positive experiences for their children or their interaction 
with teachers. 

Programmatic Factors that Influence Children’s Learning 

Parents identified many aspects of the Words Work! program that they feel help their children 
learn. They mentioned the overall emphasis on leaning (as opposed to day care), the importance 
of routines, and the opportunity to socialize. They also cited several specific classroom-based 
practices and activities that help their children learn, including: 

• Bringing books home; 

• Recitation and repetition; 

• Hands-on activities; 

• Drawing and coloring; 

• Computers; and 

• Praise from teachers; 

While parents reported being pleased with their children’s experience in general, some did note 
specific areas where they felt the program could improve. (Several facilitators noted that parents 
seemed reluctant to comment negatively on the program). 

Discipline was one area of concern for several parents. Some commented that classes were 
sometimes disrupted by unruly children. Others mentioned that they felt their child was being 
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bullied. Some did note that they felt the teachers were capably handling these situations, while 
others questioned whether or not enough was being done. 

In addition to discipline issues, parents shared the following concerns about the program: 

• Too much emphasis on play rather than learning; 

• Children come home with dirty clothes; 

• Materials are only in English; and 

• Children haven’t mastered specific skills like holding a pencil or 
■ reco gnizin g letters and numbers. 

Family Involvement 

Parents identified many ways in which they and the rest of their families are involved in their 
children’s learning. Many indicated that they imderstood the importance of a family focus on 
learning, but some commented that they did not have time to help their children as much as they 
wanted. Others shared that it was not their job to help with learning either because it was the 
school’s responsibility or language or literacy levels limited their ability to directly support 
learning. 

Parents reported being most frequently involved in the following activities with their children: 

• Making up games involving letters and numbers; 

• Reading and telling stories, in English and home languages; 

• Practicing writing letters and numbers; 

• Practicing counting; 

• Singing songs; and 

• Watching educational television shows. 

Many parents noted that they foimd informal opportunities to fit these activities into 
their daily schedules with their children, especially during car or bus rides. Several 
parents said that they used signs to point out letters to their children or incorporated 
signs into literacy games they made up and played while travelling. 

Words Work! Helping Parents to Help Children Learn 

When asked about specific Words Work! activities they were involved in, parents most 
fi"equently participating in the Book-in-a-Bag program, receiving the center newsletter, and 
attending center meetings. While parents at all sites expressed appreciation for the literacy home 
visitors also. Most parents seemed unaware of family math activities (perhaps they did not 
recognize the name) and did not participate in parent workshops. 

Many parents indicated that staff made them feel welcome to visit the classroom and volimteer at 
any time and made them feel welcome when they came. However, they cited various obstacles 
as preventing or limiting their involvement: limited education, work schedules, language 
barriers, financial problems, and transportation issues. Several parents also noted that meeting 



times were inconvenient and that allowing them to bring babies to meetings would increase 
attendance. 



Table 2: Word 


s Work! Activities Identified By Parents as Most Useful 


Activity 


Site 


Bigelow 


Baker 


McDonough 


St. Clair 


All sites 


Book-in-the- 

Bag 


♦ 


♦ 


♦ 


♦ 


♦ ♦♦♦ 


Center 

newsletter 


♦ 


♦ 


♦ 


♦ 




Parent-child 

home 

activities 


♦ 






♦ 




Monthly 

center 

meetings 


♦ 


♦ 


♦ 


♦ 




Family math 
activities 












Parent 

workshops 












Parent 
training in 
ECFE classes 








♦ 


♦ 


Parent-child 

fieldtrips 


♦ 


♦ 




♦ 




Teacher home 
visits 




♦ 


♦ 


♦ 




Literacy 
home visitor 
visits 


♦ 


♦ 


♦ 


♦ 


♦ ♦♦♦ 


Head Start 
advocate 
home visits 


♦ 






♦ 




Parent 

volunteer 

opportunities 








♦ 


♦ 



Suggestions for Improvement 

While most parents reported being generally pleased with Words Work! and the Head Start 
program, some did have specific suggestions for improvement. Many of these suggestions were 
parent-focused, such as having workshops and classes for parents to help them learn English, 
acquire their GED, or drive a car. Parents also continued to request that they receive more oral 
and written feedback on their children’s progress. Other suggestions included: 
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• Concentrate more on numbers and/or math skills; 

• Have all-day sessions for children, five days a week; and 

• Send home some Spanish-English translations for non-Spanish speaking parents to help 
their children learn more Spanish words. 
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Words Work! End of Year Staff Reflections 2001-02 
June 7, 2002 

Part I: Impact 

Impact on Children 

Words Work! staff consistently reported that the program has had a positive impact on the 
children this year. Several staff members said this was the result of the classroom environment, 
which was more focused on literacy skills, more structured, and reflected higher expectations for 
children. This environment resulted in an increase in specific skills. Staff reported more children 
performing the following skills: 

• writing their own names; 

• counting, knowing letters and sounds; 

• increasing their vocabulary; and 

• rhyming and counting word parts. 

Staff also noted improvements in children’s attitudes towards learning. One staff person noted 
that the heightened classroom structure led to increased self-esteem and confidence. Others noted 
that children are: 

• excited to learn literacy skills; 

• volunteering to read books on their own; 

• independent and knew their routine; and 

• able to concentrate better and listen more. 

Impact on Instruction 

Staff members agreed that Words Work! has had a positive impact on instruction. Many noted 
that teaching is more intentional this year. Teachers have had more formal training and were 
described as more confident, creative, and focused. They are also able to successfully incorporate 
the Words Work! goals into the classrooms. One staff person commented that assessment results 
influence how teachers plan their weekly lessons. A typical statement was: “We had a greater 
awareness with planning to integrate our goals into the curriculum.” 

Impact on Parents/Families 

Staff members also agreed that Words Work! has had a positive impact on families this year. 
Specifically, resources such as the literacy home visitor enable parents to participate more in 
their children’s education. Also, staff reported an increase in communication this year between 
families and school staff. Newsletters were noted as an important communication tool. Formal 
and informal communicating took place at parent meetings, field trips, and focus groups. 

Several staff members commented that parents are more aware of the program this year and that 
their activities reflect this awareness as well as an interest in their children’s progress. Staff 
reported that parents are: 

• asking teachers questions; 
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• asking for ways to help their children; 

• coming into classrooms to help children print names; and 

• doing more home activities. 

Impact on RAP/ Head Start or MLC 

A positive impact on RAP/Head Start and MLC was also noted. Staff commented that more 
people are becoming aware of Words Work! and that Head Start’s reputation has improved. 

They also noted that organizational changes such as the creation of the mentor/center manager 
has enabled them to spend more time both planning and with children. The cluster concept was 
also cited as enabling information sharing throughout the program. 

PART II: Strategies that Make a Difference 

What Works for Children 

In general, staff members agreed that integrating SPARK concepts into the classroom helps 
children succeed at learning. Repeating these concepts - as well as repeating stories and 
vocabulary words - also helps chil^en leam. Staff also commented that children benefit from 
routines and fi-om teachers who are able to find “teachable moments” throughout the day. Staff 
members also noted more specific classroom strategies that work for children, including: 

• concept words appearing throughout the room; 

• index box with all the words in it; 

• eye-level presentations; 

• practicing rhyming in silly ways; 

• using fun and humor; 

• si gnin g in each morning and afternoon; 

• asking open-ended questions; 

• using music; 

• allowing children to role play; and 

• printing concept words at the table. 

What Gets In the Way for Children 

Staff noted several impediments to children’s learning. Some of these are related to the Words 
Work! program. For instance, having too many adults in the classroom and allowing outside 
people to come though classrooms were cited by a few as impeding the learning process. Also, 
staff turnover was noted to have a negative impact on learning. One staff person commented on 
an unsafe environment. 

Many other impediments to learning are not related to the program itself, but could potentially be 
addressed in the classroom. These include: 

• behavior problems; 

• hygiene problems; 

• attendance; 
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difficulty focusing; 

fear of failure and new ideas; and 

lack of parental involvement. 



What Works for Instruction 

Staff noted that many aspects of the Words Work! program contribute to successful instruction. 
Many agreed that teacher mentors and literacy home visitors provide a valuable resource to 
teachers. Sharing strategies between teachers and across sites was also cited as beneficial. 
Increased teacher training led to well-planned lessons, and some staff commented that a balance 
between smart planning and the ability to be flexible contributed to successfully meeting the 
needs of all students. One center manager commented that writing out goals and posting them 
reminds teachers to stay focused. 

What Gets In the Way of Instruction 

Staff reported that impediments to instruction are divided in the same manner as impediments to 
children’s learning. Some are directly related to the Words Work! program while others result 
fi'om characteristics of children and their families. In the first category, certain issues are staff- 
based. Two staff members reported not having enough time for preparation or reflection. Two 
center managers commented that staff turnover hinders instruction. Also mentioned were staff 
inconsistency, lack of bilingual skills and cultural knowledge, and failure to follow routines. 
Other issues are program- wide, including poor classroom management, lack of communication 
between staff and other support services, and limited materials. None of these issues, however, 
were cited by more than one staff member. 

Other impediments to instruction concern children’s and families’ characteristics, and closely 
resemble the reported impediments to children’s learning. These include behavior problems, 
poor attendance, language barriers, family mobility, and lack of family interest. 

What Works for Families 

Staff reported several strategies that worked for involving families this year. Many of these had 
to do with communication. Factors that staff said improved communication between themselves 
and parents this year included: 

• sending home weekly newsletter; 

• translating materials and meetings into parents’ home languages; 

• daily contact with parents; 

• parents knowing how to ask better questions; 

• sending home more books; and 

• sending home specific activities for families to do at home. 

Staff also reported that specific aspects of the program contributed to parents’ increased 
involvement this year. The literacy home visitor was most frequently cited as a positive factor 
who both enhances co mmuni cation and reinforces classroom activities. Having SPARK stories 
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available in the library and integrating Words Work! goals into parent-teacher communication 
were also cited as contributing factors. 

What Gets In the Way of Families’ Involvement 

Certain factors continue to hinder increased family involvement in the Words Work! program. 
The most-cited of such factors was a language barrier. Family demands including other children, 
work schedules, and finances were also mentioned as hindrances to increased parental 
involvement. Staff also noted that families’ attitudes about school and learning play a role in 
their involvement. Welfare-to-work time limits have also had an impact: newly working parents 
have less time for home visits and volunteering opportunities. 

Suggestions For Improvement 

Staff members had a number of suggestions for improving the Words Work! program. May of 
these had to do with fijrther improving communication between families and schools, including: 

• improving forms of communication besides newsletters; 

• translating more paperwork into home languages; 

• using bus drivers more for transporting paperwork home; 

• using parent meetings for more meaningfixl connections rather than 
simply updates; and 

• having a Words Work! workshop to explain program to parents. 

Other suggestions had to do with increasing cultural awareness, including: 

• increasing staff s ability to connect with families of different cultures; 

• encouraging more parent involvement fi’om all cultures; and 

• training staff more in cultural childrearing. 

Other suggestions were related to professional development, including: 

• scheduling agency-wide trainings and orientations; 

• defining best practices and how to implement them; 

• better using professional development days; 

• training staff in child portfolio presentation; 

• adding a floating staff person to each center; and 

• partnering with St. Paul Technical College to put student aides in 
classrooms. 

Program Impacts on Work 

Many staff members mentioned ways that they are now performing their jobs differently as a 
result of the Words Work! program. The most-cited example was that teachers are now teaching 
with intentionality. Staff also mentioned that they are getting useful feedback form assessment 
data and focus groups. They also reported that both RAP and MLC have become more aware of 
and focused on children’s literacy. 
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Influence of Co-Workers 



Words Work! staff generally agreed that their influence on their non-Words Work! colleagues 
has been subtle. Several commented that there is curiosity and interest in the program in non- 
WW sites. One mentioned that some of this produces envy at other sites. Others remarked that 
non-WW sites do not have the resources to perform WW tasks, or they feel that they already 
perform these tasks but avoid the WW paperwork. 
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Words Work! 

2002 Strategies for Learning 



Staff at all four Words Work! sites use a variety of classroom strategies to assist in achieving the 
program’s literacy goals. The following list was culled from reports of classroom observations 
at the four sites. 



Classroom Environment Strategies 


WW! Goal 


Strategy or Activity 


1. Print awareness- 


• Writing table 

• Use classroom sign-in and then have kids take their names off 
and put in envelope before they go to play time 

• Sign up sheet to use the computer 

• Children’s written letters — ^posted on bulletin board — some 
letters were waiting to be mailed 

• Children’s writing samples posted 

• Kids write their names when they come in 


2. Vocabulary 


• “Life in Rain Forest” bulletin board with names like canopy, 
forest floor, etc. 

• Names on object in house 

• Story words on poster paper 

• Posters with sentences 

• SPARK concepts and title of story on white board 

• Words for “Twinkle, Twinkle Little Star” displayed 

• Labels on items in the class 


3. Phonological 
Awareness 


• Practice ABCs and sounds in group 

• Dictated stories, rhymes and songs on walls, desks, shelves 


4. Alphabet 

Name activities: 


• ABC song during lice check 

• Alphabet cards 

• Sign language poster 

• Wooden alphabets 

• Children choose letters to sit on during transitions and teachers 
have them practice identifying the letters 

• Magnetic alphabet 

• Alphabet rug 

• Making letters with the play dough 

• Color magnetic alphabets on the white board 


• Playuig alphabet bingo 

• Alphabet posted throughout classroom 

• Students try to recognize their names on place cards before 
going to play area 

• Names on Velcro apple 

• Velcro name cards 

• Construction paper name cards 
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• Paper with kids names by coats 

• Children write their names on their projects 

• Itsy bitsy spider display on heating pipes; spiders with kids 
names on them to help kids get in line and know how to line 
up 

• Children whose names start with certain letters get up and 
wash hands first 


5. Nvimbers 


• Poster with numbers 

• Nvimbers on wall and writing table 

• Numerous counting activities, such as counting verses in 
songs 

• Counting parts of snake 

• Large numbers over the window 

• Walk around block to count telephone poles 

• Counting in languages other than English 

• Use of feathers, marbles and other props for teaching counting 


6. Shapes 


• Border of shapes around bulletin board 

• Have children sit on different shapes on rug and call them 
according to shape 


7. Language 


• Role playing with animals, with students “teaching” animals 
what they are learning 

• Wide range of books 


8. Parent-Family 
Involvement 


• ABC book at-home activity 

• Parents visit, either to observe or help out with activities 

• Pictures of kids with parents on wall 

• Family tree with leaves and pictures 

• Compliment and complaint forms available for parents 


9. Multicultural and 
Lingual 


• Multi-cultural music 

• Some classroom object names in Spanish and English 

• Some instructions given in Spanish 

• Borders with different cultmes 

• Multicultural dolls 

• Poster of shapes in English and Hmong 

• Colors poster in Hmong 

• Word posters in English, Spanish, and Hmong 
. • Listen to songs in Spanish 

• Information translated into Somali for parent — ^but parent did 
not read Somali 

• Learning to count in English and Spanish in one class 

• Bilingual, multicultural books 


10. Multi-sensory, 
Hands on 


• Colors and manipulatives on science table 

• Sand table 

• Blowing bubbles 

• Follow the leader going under/over/aroimd objects 





• Science center 


11. Arts as follow-up 


• Self-portraits 

• Children made fish as part of a school of fish 

• Children’s faces with yam and names 

• Fall leaves in bright colors with student names 

• Dressing up in adult clothes 

• Watercolors 

• Made telephones and practiced talking 


10. Colors 


• Colors poster 

• Names of colors at science table 

• Asked children to name colors during meeds or small group 
activities 

• Asked children to name colors of objects in SPARK stories 

• Pasted orange paper pieces on pumpkin 

• Kids line up on feet that match the color of their coats 



SUMMARY OF WORDS WORK! CASE STUDIES 
YEAR 1 TO YEARS 
June 2002 

Prepared by Chia Youyee Vang and Victoria Amaris 
INTRODUCTION 

From Year 1 to Year 3, a total of nine children have participated in the case studies. During the 
first year of implementation, three children were randomly selected and three new students were 
added during each subsequent year. The demographics of the students are: three Hmong, three 
Latino, one Afiican American, one Somali, and one Caucasian. 

The new case studies for year three included student assessment results, interviews with the 
child’s parent(s) in the fall, winter and spring and classroom observations. Informal 
conversations with teachers and activity sheets also informed the process. For the follow-up 
cases, interviews with the children’s parent(s) took place in fall 2001 and in spring 2002. All 
parents were asked to share information about their children’s interests, their involvement in 
their child’s school, activities they did with their children at home, what is working well at Head 
Start and what Head Start might do better. When possible, the evaluators would ask the case 
study children questions. 



GENERAL OBSERVATIONS 

Several overall observations regarding the impact of Head Start on children’s preparedness for 
school were noted, including: 

• Preparing most children well for kindergarten and beyond; 

• Impacting children’s behavior and contributing to children’s maturity; 

• Supporting learning by providing strategies that are adaptable to children regardless of 
cultural backgrounds; 

• Increasing early literacy skills of children to prepare them to be successful readers and 
learners when they are in school; and 

• Increasing the family’s role in the development and delivery of high quality early literacy 
experiences for their children. 

CASE STUDY PROFILES 

New Case Studies for 2001-02 

From Year 3 “Youa.” Youa is the fourth of six children. The mother does not work outside of 
the home. The father works at a company where he is an assembler. They have been in the U.S. 
for 12 years. 

All of the interviews were conducted with the mother. The mother said that enrolling her 
daughter at Head Start is important in preparing her for kindergarten. She has seen significant 
changes in her child. She stated that her child knew very little English before going to Head 
Start. Toward the middle to the end of the school year, the child could speak English with her 
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older siblings. The mother credited her child’s ability to count and print her name to Head Start. 
She said, “Her dad and I didn’t teach her any of this. She knew how to count in Hmong from 1- 
10, but she did not know how to say it in English. It seems that she would learn certain things in 
school, and then older siblings would help her at home. Sometimes the kids would tell each 
other the stories that they were learning at school. [Youa] tells her older siblings her stories from 
Head Start.” 

The mother does feel that her daughter is very shy, so Head Start has been good for her. She is 
at least interacting with kids who are not relatives. The family does not have very many 
activities that they do together. At home, the children play with their toys and draw pictures. 

The father occasionally read to the children when he has spare time. However, the children seem 
to keep themselves busy with their own activities. 

The evaluator read Goodnight Moon to the child in the fall. The child was able to point to a few 
of the different items in the story and named them. However, she did not know how a book 
worked. When asked what she liked most at school, she said singing and stories. During the 
second and third interviews, the child clearly knew how a book worked. 

In fall 2001, Youa received a “yes” on many of the SPARK activities except, naming yellow, 
opposites, song, same, size, sequence, body, address, position, who, what if, own, feelings, 
similarities, group, events, answer, and sing. She got a 6 in book, a 2 in story and zeros on the 
other English phonological assessments. When the phonological assessments were given in her 
home language, she got a 7 on book, a 3 on word, 1 on story, and zeros on the rest. On the 
Boehm, she was in the 75th percentile when the instructions were given in her home language. 
She got 3 on the ABCs and was rated a 6 on attempt to print. 

In spring 2002, Youa received "yes" on nearly all her SPARK activities except in short songs, 
address, and positional words. On the phonological assessments, she received a 7 (7 in home 
language) ) on book, a 3 (4) on story, 0 (3) on Words, 0 (0) on sound and rhyming, 6 (0) on 
syllable. On the Boehm, she was ranked in the 50th percentile in English and 90th percentile in 
home language. She knew 25 letters of the alphabet and received a score of 6, the highest 
score, on attempt to print. 

From Year 3 “Lupe.” Lupe lives with her mother, father and brother. Her family has been in 
the U.S. for four years, all of which have been in St. Paul. Her mother is currently a housewife. 
Her educational background is completion of a high school degree. This is Lupe’s second year 
at Head Start. 



Status of Child 

Lupe is very sociable, articulate and happy. She seems to like everything. During the fall 
interview, she preferred not to share and tended to scream to get her way. She liked to draw. 
During the second and third interviews, she was not bratty as previous and shared more 
willingly. She was also more polite and obedient and seemed more mature. 

In the fall, she counted up to 10 and then skipped to 20, sometimes mixing Spanish and English. 
In the winter, she counted to 15, but could not identify separate numbers. During the spring, she 
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counted to 20 without skipping any numbers. Lupe can write her name and identify some other 
letters of the alphabet. However, her mother felt that she was going slower than her older 
brother. Lupe liked to copy print materials and knew most of her colors by the end of the year. 
She learned both from home and school. 

At home, her parents mostly speak to her in Spanish. When Lupe speaks to her parents, she 
sometimes spoke English. Also, when talking to her brother, she would speak English. 

However, her mother encourages her to speak Spanish. 

Head Start Involvement 

Lupe’s mother stated that she had enrolled her daughter in Head Start because she felt her 
daughter needed to know people, other than her family. She felt that this would not only help 
improve her behavior, but also, it would prepare her for school and help her learn English. Her 
mother knew her teacher and had met with her. She talked about being able to speak in Spanish 
with her child’s teacher. She used everything that is sent home to help her child learn, including 
bilingual books, and activity sheets. Interesting, the mother said that she liked the bilingual 
books, but her daughter preferred to be read to only in English. 

When she has concerns, the mother speaks directly with her child’s teacher. Having a bilingual 
teacher is helpful to her. She also appreciated that the librarian speaks Spanish. What the 
mother valued most about her child attending Head Start is that the teacher is very good and 
helps her daughter a lot. She said that her “child demands a lot of patience and teacher and 
teachers have more patience than mothers. Teachers can get children to do things better than 
mothers.” The mother credited her child’s teacher for helping to improve her child’s attitude and 
behavior. She is very satisfied with her child’s progress and feels that Lupe is well prepared. 

Activities at Home 

At home, Lupe likes to read books, tell stories, and works on puzzles. She also likes to help with 
cooking and setting the table. When helping to cook, she counts things. Toward the end of the 
school year, she would write her name, cut, draw, cook, pretend to read books and memorize 
books, only in English. Activities that Lupe and her mother do together include, reading, 
cooking, drawing, counting activities, play outside, go for walks, and visit the children’s 
museum. Her mother felt that reading, counting and talking about things help her daughter learn. 
Reading is something that the family is able to do regularly. The mother, father, and older 
brother all help Lupe read. 

The mother identified several activities that support her child’s learning. They included helping 
her to focus on writing, having a scheduled time for reading, and listening to the child when she 
wants to share her ideas. What is difficult for the mother is that she has difficulties reading in 
English, but her daughter prefers to be read to in English. She is interested in obtaining a library 
card, but does not know how to do so. The evaluator explained the process to her. The mother 
said that there used to be a home visitor, but there has not been anyone coming for months. She 
liked having the home visitor come and missed that interaction. 
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Student Assessment 



In fall 2000, Lupe received a “yes” to all the SPARK activities. She was emerging in all of her 
shapes. On the Boehm, she was in the 16**’ percentile. The child was never tested in her home 
language. She received a 2 on her ABC’s and a 4 on Attempt to print. 

In spring 2002, she received a “yes” on all SPARK activities except on “Address” and “Ask 
What If’ (new SPARK items); 7 on Book, 3 on Story, 2 on Syllable, and 0 on Word, Rhyming, 
and Sound. She was in the 28th percentile on the Boehm (in both home and English languages). 
She knew four letters and received a 6 on Attempt to Print. 



From Year 3: “Fahima.” Fahima and her mother came from Somalia and have been in the U.S. 
for five years, all of which have been in St. Paul. Her father and siblings are still in Africa. 

Status of Child 

According to her mother, Fahima likes speaking English. She can be both social and fearfixl. 

She likes to color, read, count and say the alphabet. She counts to 1 0. At home, they speak both 
Somali and English. Her mother said that she learned the skills both from home and school. Her 
English skills improved a great deed during the year. She is focused and writes her name well. 

By the end of the year, she can write titles of books, make books and seems to like to handle 
them. Her teacher said that she is a perfectionist and is sometimes hard on herself. 

Head Start Involvement 

Fahima’s mother’s reason for enrolling her in Head Start was that her daughter needed to learn 
more things, improve her English skills to prepare her for kindergarten. The mother does know 
her daughter’s teachers. She mostly used die books and newsletters to help support her 
daughter’s learning. What the mother valued most about her daughter attending Head Start is, 
“She has become better at expressing love to family. She obeys better and plays better with 
other children. She also reads better.” The mother would like to see more computers available 
at Head Start and more work activities. However, she is very satisfied with her child’s overall 
experience. 



Activities at Home 

At home, Fahima does the following activities: color, count, read stories, attend cultural 
activities in the Somali community, and other activities to learn about her culture. The mother 
does help her child with homework, but because she is now working full-time, it is difficult to do 
many things with her child. She does have some people in the Somali community who helps her 
daughter with reading books. 



Questions for Child 
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When asked what she is learning at school, Fahima said that she was learning her numbers, 
colors, playing, and computers. She said that she liked everything, especially coloring and 
playing. She said that she did not like puzzles and books. However, classroom observations and 
conversations with her teacher reveal that she does like books and puzzles. 

Student Assessment 

In the fall, she received mostly “yes” on the SPARK activities. She had a good understanding of 
a book and received a 6. She received a 2 for story, 3 for word, 0 for rhyming, 2 for syllable, and 
2 for soimds. She was in the 17* percentile on the Boehm. She knew 1 1 letters and received a 4 
on attempt to print. 

In spring 2002, Fahima received “yes” on all SPARK activities; 6 on Book, Word, and Syllable, 

4 on Story and Sound, and 2 on Rhyming. She was in the 12th percentile on the Boehm, knew 
24 letters of the alphabet and received a 6 on Attempt to print. 

Follow-Up Cases 

From Year 2 “Neng.” Neng is attending HOPE Academy charter school for the 2001-02 
academic year. She is in first grade. As the yoimgest child, she receives help fi’om her older 
siblings. Her father works first shift while her mother works the second shift. According to the 
mother, the family is in a very stressful situation with both parents working and hardly ever 
seeing each other. The mother feels that she has very little time to spend with her children. 

Since she works in the evening, she is not able to be with the children when they come home at 
ni ght She caimot attend conferences so her husband has to do so. She stated that her husband 
was satisfied with their daughter’s progress in school. Her teacher thinks she is doing well, but 
is still quite shy. 

The mother said that she had seen great improvement in her daughter’s ability to speak English. 
She gives her teacher a lot of credit and expresses concerns for her own lack of time to be with 
her children. She stated, “We barely have time to eat together once or twice during the entire 
week. There is no time in our live for activities, especially during the cold season.” 

When asked to recite the letters of the alphabet, the child could do so. The evaluator wrote the 
letters of the alphabet and asked the child to read the letters out of order. The child was able to 
recognize all letters of the alphabet. The mother credited her husband in addition to the older 
siblings and her teacher for this progress. When asked what she liked most at school, she said 
that she liked s ingin g songs and coloring. In the spring 2002 interview, the child could soimd 
out several words, such as “she,” “he,” and “go.” 

From Year 2 “Miguel.” Miguel’s father has always lived in the U.S. while his mother is from 
Mexico. His parents have a licensed home day care business. She is a housewife while he works 
as a security guard. His mother has a high school diploma while his father completed 10* grade. 
He has two older brothers. 
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Status of Child 



Miguel is still very sociable and likes to see friends and family members. He has changed in that 
he is more talkative to everybody. He likes homework and is proud. He does not like to sit in 
one place or listen to the instructor. He can now count to 100, with a little help in tens. He 
knows his ABC’s and colors. The parents teach him at home and he also learned from school. 

At home, the parents mostly speak Spanish to him. The child usually responds both in English 
and Spanish; however, English seems to be preferred by the child unless in an environment that 
only Spanish is spoken. 



School Involveinent 

Miguel was in all day kindergarten, so it took some time to get used to it. The parents know their 
child’s teacher and feels that she is a good teacher. They usually use the materials or homework 
that Miguel brings home to help him learn. The parents feel comfortable talking directly to their 
child’s teacher if they have concerns or questions. They said that their son really likes to be in 
school and that he is learning and being responsible. They are quite happy with their son’s 
progress in school and like the fact that there are a lot teachers. 

Activities at Home 

According to his parents, Miguel likes to be active. He now plays the piano and likes to color a 
lot, especially shapes and patterns. Activities that they do with their son include homework, 
talking about different things, and going to church. His parents think that the bible stories to 
teach their son what is right and wrong. Reading at night and doing homework together also 
help to their son leam. They read to their son three to four times per week. The parents try to 
“make sure he completes his homework and discuss what happened in school to help his 
memory.” Some things that gets in the way of their son’s learning is that he wants to be outside 
too much, the yoimger children bother him and that he likes video games a lot. 

Questions for Child 

Miguel said that he was learning a lot of things at school. He mentioned sharing, making pattern 
snakes and books. What he liked most about school was going outside, breakfast, eating ice 
cream and snack time. He said that he did not like math. 

From Year 2 “Jenny.” Jenny’s family has lived in St. Paul for eight years. There are three 
children in the family and she is the youngest. Her mother’s educational background is one year 
of college. Her occupation is housewife/child care. The family has moved away before the end 
of the school year, thus the child v^ll no longer be participating in the project. 

Status of child 

Jenny appeared to shy at first, then became “a talker.” She is friendly, outspoken and caring. 

She said that she did not like yelling. She likes to read and write as well as make things for other 
children. She is exceptionally good, but sometimes gets angry with her older brothers. In the 
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fall, she counted up to 15 and in the spring, she could count to 20. She learned her ABCs, 
counting and colors at home, day care and at Head Start. The primary language is English. 

Head Start Involvement 

Jenny’s mother indicated that she enrolled her daughter in Head Start to help prepare her for 
kindergarten. She stated, “So she can be aroimd other children. Be in a more learning 
environment and be ready for kindergarten.” The mother knew her child’s Head Start teacher by 
name and had met with her. Throughout the school year, she used materials provided by Head 
Start to help her child learn, such as SPARK stories, newsletters, and sheets. When she had 
questions, she would contact the Head Start center manager. She did, however, expressed some 
concern about the center manager needing to be more available. What the mother valued most 
about her child being at Head Start is “that she loves being there. She’s sad when not there.” 
Additionally, Head Start is “preparing her for the next grade, interacting with others...” 

Although she is very satisfied with her child’s experience at Head Start, Jenny’s mother said that 
she would like parent meetings to also be held in the evening so she can attend. This last school 
year, the meetings were switched to day time, and she could not attend any of the them. 

Activities at Home 



Jenny’s mother said that at home, she eats, watches some television, and sometimes plays with 
her Barbie dolls; however, she would mostly read and write. During the spring interview, her 
home activities included computer work with the above activities. Her mother felt that 
practicing to write her name, numbers, reading and using flash cards help her child to leam. She 
read to her child four to five times a week. Jenny’s older brothers also help her with reading. 
Some challenges that get in the way of her helping her child’s learning included household 
duties, personal issues, and other kids. Household chores make her tired, which makes it 
difficult for her to do more. There are a few activities outside of school that help her daughter 
leam. They attend church where her daughter is able to read books and look at Christian news. 
Additionally, the family goes to the library. 

Questions for Child 

When asked what she was learning in school, the child responded that she was learning her 
ABC’s and reading a book. She said that she liked to write her ABC’s most at school. She 
seemed to like everything at school, with the exception of taking a nap. 

From Year 1: “Rosita.” The following case study is based on activities carried out over the past 
year as part of the follow-up case study. Included were two in-depth interviews with the parent, 
and questions for the child; reading time with the child. No observations of child in kindergarten 
classroom were made, nor was there access to her school data. 

“Ana,” the mother of the child is a 41 year-old Latina who immigrated from Mexico 15 years 
ago. She and her family have lived in St. Paul for four years. The family includes three 
children, with the case study child being the oldest, at nearly six years of age. There has been a 
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move to a new home, much larger and nicer. It is also one block from where Rosita goes to 
kindergarten. The husband does not appear to be part of the household at this time. Spanish is 
the primary language at home. 



The chUd in kindergarten 

This was Rosita’s first year in kindergarten. This year she continues to be sociable, inquisitive 
and very much a communicator. She is more able to express her feelings and is also more 
patient and focused in her attention. Her listening and reading skills have also developed more. 

Her mother reports that she has performed very well in school and has benefited from being in a 
bilingual classroom. Her Spanish is clearly stronger than before although she had previously 
been exceptionally bilingual. She has learned more about her two cultures and is a proud Latma. 
Her teacher has helped her to learn to be calmer and to improve her behavior. 

The child is also very proud of her progress and excitedly shared the many things she does well 
at school: reading, drawing and coloring inside the lines, “cutting very well”/ “cotando muy 
bien.” Without prompting, she offered to read to the evaluator at the last interview. She read 
from own book and soimded out letters when she needed to in Spanish. 

In summary, it appears that this child has been prepared well for learning through her Head Start 
experience. It nurtured her love for books and learning and also provided the focus she needed, 
giving her a solid foimdation for future learning. The fact that she is in a bilingual kindergarten 
classroom is an added value to maximize her learning in two languages. 

In her own words, Rosita excitedly told the evaluator, “The class can be interesting and I can do 
things very well.” 

From Year 1: “Debbie.” Debbie is one of five children and lives in a two parent household, 
where she is the yoimgest. Her mother is a housewife and does elder care on a part-time basis. 
The mother is a 3 1 year-old white woman who has lived in St. Paul her entire life. Her 
educational backgroimd is completion of 1 1* grade. 

Debbie finished first grade in spring 2001 . Her mother reports that she is doing very well. She 
remains sociable and has many fiiends. She is physically large for her age, which her mother 
feels brings its own challenges. She likes to be heard and express her opinions. She can now 
coimt past 100 and likes numbers. 

She enjoys reading and reads everyday. She reads labels, the cartoons in the paper, large print 
books, books from school and the library. In the car, she writes letters and now enjoys using her 
computer skills. But, some reading is difficult for her and her mother feels she should be reading 
at a higher level. 

From Year 1 “Tou.” Tou still attends Maxfield Elementary school and is now in second grade. 
Both parents were present for the fall interview and the spring interview was conducted only 
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with the mother. The mother lost her job in September, and it took her several months to find 
another one. The father still works a second shift job as a machinist. 

The father was still concerned about the progress his son was making in school. The child seems 
to xmderstand things, but often does not respond when asked questions. His teacher feels 
confident that he is moving in the right direction, but the father stated that he felt his son was not 
getting the help he needs. 

At home, there are not much activities for the child except watch television and play games with 
his brother and sister. 0n weekends, the family usually spend time at relatives’ houses for 
different cultural events. The child then plays with his many cousins. 

The evaluator read the book. The Paper Crane, to the child in English. She then read it in 
Hmong language. His siblings came over and listened with him. She also asked him questions 
about the fi’ont of the book, the back and what does the author of the book do. The child always 
responds in Hmong language even when a question was asked in English. He said that he liked 
drawing most at school and playing with his friends. 

During the spring interview, the child printed all the letters of the alphabet and was able to speak 
more fluently in English. He still mixed Hmong words in his sentences. 
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